{"title": "Interlaced Greedy Algorithm for Maximization of Submodular Functions in Nearly Linear Time", "book": "Advances in Neural Information Processing Systems", "page_first": 2374, "page_last": 2384, "abstract": "A deterministic approximation algorithm is presented for the maximization of non-monotone submodular functions over a ground set of size $n$ subject to cardinality constraint $k$; the algorithm is based upon the idea of interlacing two greedy procedures. The algorithm uses interlaced, thresholded greedy procedures to obtain tight ratio $1/4 - \\epsilon$ in $O \\left( \\frac{n}{\\epsilon} \\log \\left( \\frac{k}{\\epsilon} \\right) \\right)$ queries of the objective function, which improves upon both the ratio and the quadratic time complexity of the previously fastest deterministic algorithm for this problem. The algorithm is validated in the context of two applications of non-monotone submodular maximization, on which it outperforms the fastest deterministic and randomized algorithms in prior literature.", "full_text": "Interlaced Greedy Algorithm for Maximization of\n\nSubmodular Functions in Nearly Linear Time\n\nAlan Kuhnle\n\nDepartment of Computer Science\n\nFlorida State University\nTallahassee, FL 32306\nakuhnle@fsu.edu\n\nAbstract\n\nA deterministic approximation algorithm is presented for the maximization of\nnon-monotone submodular functions over a ground set of size n subject to cardi-\nnality constraint k; the algorithm is based upon the idea of interlacing two greedy\nprocedures. The algorithm uses interlaced, thresholded greedy procedures to ob-\n\n(cid:1)(cid:1) queries of the objective function, which\n\ntain tight ratio 1/4 \u2212 \u03b5 in O(cid:0) n\n\n\u03b5 log(cid:0) k\n\nimproves upon both the ratio and the quadratic time complexity of the previously\nfastest deterministic algorithm for this problem. The algorithm is validated in\nthe context of two applications of non-monotone submodular maximization, on\nwhich it outperforms the fastest deterministic and randomized algorithms in prior\nliterature.\n\n\u03b5\n\n1\n\nIntroduction\n\nA nonnegative function f de\ufb01ned on subsets of a ground set U of size n is submodular iff for all\nA, B \u2286 U, x \u2208 U \\ B, such that A \u2286 B, it holds that f (B \u222a x) \u2212 f (B) \u2264 f (A \u222a x) \u2212 f (A).\nIntuitively, the property of submodularity captures diminishing returns. Because of a rich variety of\napplications, the maximization of a nonnegative submodular function with respect to a cardinality\nconstraint (MCC) has a long history of study (Nemhauser et al., 1978). Applications of MCC\ninclude viral marketing (Kempe et al., 2003), network monitoring (Leskovec et al., 2007), video\nsummarization (Mirzasoleiman et al., 2018), and MAP Inference for Determinantal Point Processes\n(Gillenwater et al., 2012), among many others. In recent times, the amount of data generated by many\napplications has been increasing exponentially; therefore, linear or sublinear-time algorithms are\nneeded.\nIf a submodular function f is monotone1, greedy approaches for MCC have proven effective and\nnearly optimal, both in terms of query complexity and approximation factor: subject to a cardinality\nconstraint k, a simple greedy algorithm gives a (1 \u2212 1/e) approximation ratio in O(kn) queries\n(Nemhauser et al., 1978), where n is the size of the instance. Furthermore, this ratio is optimal\nunder the value oracle model (Nemhauser and Wolsey, 1978). Badanidiyuru and Vondr\u00e1k (2014)\nthe approximation ratio, while Mirzasoleiman et al. (2015) developed a randomized (1 \u2212 1/e \u2212 \u03b5)\napproximation in O(n/\u03b5) queries.\nWhen f is non-monotone, the situation is very different; no subquadratic deterministic algorithm has\nyet been developed. Although a linear-time, randomized (1/e\u2212 \u03b5)-approximation has been developed\n\nsped up the greedy algorithm to require O(cid:0) n\nby Buchbinder et al. (2015), which requires O(cid:0) n\n\n(cid:1) queries while sacri\ufb01cing only a small \u03b5 > 0 in\n(cid:1) queries, the performance guarantee of this\n\n\u03b5 log n\n\n\u03b5\n\nalgorithm holds only in expectation. A derandomized version of the algorithm with ratio 1/e has been\n\n\u03b52 log 1\n\n\u03b5\n\n1The function f is monotone if for all A \u2286 B, f (A) \u2264 f (B).\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fTable 1: Fastest algorithms for cardinality constraint\n\nTime complexity Deterministic?\n\nAlgorithm\n\nFastInterlaceGreedy (Alg. 2)\n\nGupta et al. (2010)\n\nBuchbinder et al. (2015)\n\nRatio\n1/4 \u2212 \u03b5\n1/6 \u2212 \u03b5\n1/e \u2212 \u03b5\n\nO(cid:0) n\nO(cid:0)nk + n\nO(cid:0) n\n\n\u03b5 log k\n\n\u03b52 log 1\n\n\u03b5\n\n(cid:1)\n(cid:1)\n(cid:1)\n\n\u03b5\n\n\u03b5\n\nYes\nYes\nNo\n\ndeveloped by Buchbinder and Feldman (2018a) but has time complexity O(k3n). Therefore, in this\nwork, an emphasis is placed upon the development of nearly linear-time, deterministic approximation\nalgorithms.\n\nContributions\n\n\u03b5\n\n\u03b5 log k\n\nThe deterministic approximation algorithm InterlaceGreedy (Alg. 1) is provided for maximization\nof a submodular function subject to a cardinality constraint (MCC). InterlaceGreedy achieves ratio\n1/4 in O(kn) queries to the objective function. A faster version of the algorithm is formulated in\n\nFastInterlaceGreedy (Alg. 2), which achieves ratio (1/4 \u2212 \u03b5) in O(cid:0) n\n\n(cid:1) queries. In Table 1,\n\nthe relationship is shown to the fastest deterministic and randomized algorithms for MCC in prior\nliterature.\nBoth algorithms operate by interlacing two greedy procedures together in a novel manner; that is,\nthe two greedy procedures alternately select elements into disjoint sets and are disallowed from\nselection of the same element. This technique is demonstrated \ufb01rst with the interlacing of two\nstandard greedy procedures in InterlaceGreedy, before interlacing thresholded greedy procedures\ndeveloped by Badanidiyuru and Vondr\u00e1k (2014) for monotone submodular functions to obtain the\nalgorithm FastInterlaceGreedy.\nThe algorithms are validated in the context of cardinality-constrained maximum cut and social\nnetwork monitoring, which are both instances of MCC. In this evaluation, FastInterlaceGreedy\nis more than an order of magnitude faster than the fastest deterministic algorithm (Gupta et al.,\n2010) and is both faster and obtains better solution quality than the fastest randomized algorithm\n(Buchbinder et al., 2015). The source code for all implementations is available at https://gitlab.\ncom/kuhnle/non-monotone-max-cardinality.\n\nOrganization The rest of this paper is organized as follows. Related work and preliminaries on\nsubmodular optimization are discussed in the rest of this section. In Section 2, InterlaceGreedy and\nFastInterlaceGreedy are presented and analyzed. Experimental validation is provided in Section 4.\n\nRelated Work\n\nto obtain the same ratio in O(cid:0) n\n\nThe literature on submodular optimization comprises many works. In this section, a short review of\nrelevant techniques is given for MCC; that is, maximization of non-monotone, submodular functions\nover a ground set of size n with cardinality constraint k. For further information on other types of\nsubmodular optimization, interested readers are directed to the survey of Buchbinder and Feldman\n(2018b) and references therein.\nA deterministic local search algorithm was developed by Lee et al. (2010), which achieves ratio\n1/4 \u2212 \u03b5 in O(n4 log n) queries. This algorithm runs two approximate local search procedures in\nsuccession. By contrast, the algorithm FastInterlaceGreedy employs interlacing of greedy procedures\nformulated by Vondr\u00e1k (2013), which achieves ratio \u2248 0.309 in expectation.\nGupta et al. (2010) developed a deterministic, iterated greedy approach, wherein two greedy pro-\ncedures are run in succession and an algorithm for unconstrained submodular maximization are\nemployed. This approach requires O(nk) queries and has ratio 1/(4 + \u03b1), where \u03b1 is the inverse\nratio of the employed subroutine for unconstrained, non-monotone submodular maximization; under\nthe value query model, the smallest possible value for \u03b1 is 2, as shown by Feige et al. (2011), so\nthis ratio is at most 1/6. The iterated greedy approach of Gupta et al. (2010) \ufb01rst runs one standard\ngreedy algorithm to completion, then starts a second standard greedy procedure; this differs from\nthe interlacing procedure which runs two greedy procedures concurrently and alternates between\n\n(cid:1) queries. In addition, a randomized local search algorithm was\n\n\u03b5 log k\n\n\u03b5\n\n2\n\n\fthe selection of elements. The algorithm of Gupta et al. (2010) is experimentally compared to\nFastInterlaceGreedy in Section 4. The iterated greedy approach of Gupta et al. (2010) was extended\nand analyzed under more general constraints by a series of works: Mirzasoleiman et al. (2016);\nFeldman et al. (2017); Mirzasoleiman et al. (2018).\nAn elegant randomized greedy algorithm of Buchbinder et al. (2014) achieves expected ratio 1/e\nin O(kn) queries for MCC; this algorithm was derandomized by Buchbinder and Feldman (2018a),\n\nbut the derandomized version requires O(cid:0)k3n(cid:1) queries. The randomized version was sped up in\nBuchbinder et al. (2015) to achieve expected ratio 1/e\u2212\u03b5 and require O(cid:0) n\n(cid:1) queries. Although\n\nthis algorithm has better time complexity than FastInterlaceGreedy, the ratio of 1/e \u2212 \u03b5 holds only\nin expectation, which is much weaker than a deterministic approximation ratio. The algorithm of\nBuchbinder et al. (2015) is experimentally evaluated in Section 4.\nRecently, an improvement in the adaptive complexity of MCC was made by Balkanski et al. (2018).\n\nTheir algorithm, BLITS, requires O(cid:0)log2 n(cid:1) adaptive rounds of queries to the objective, where\n\n\u03b52 log 1\n\n\u03b5\n\nthe queries within each round are independent of one another and thus can be parallelized easily.\nPreviously the best adaptivity was the trivial O(n). However, each round requires \u2126(OP T 2) samples\nto approximate expectations, which for the applications evaluated in Section 4 is \u2126(n4). For this\nreason, BLITS is evaluated as a heuristic in comparison with the proposed algorithms in Section 4.\nFurther improvements in adaptive complexity have been made by Fahrbach et al. (2019) and Ene and\nNguyen (2019).\nStreaming algorithms for MCC make only one or a few passes through the ground set. Streaming\nalgorithms for MCC include those of Chekuri et al. (2015); Feldman et al. (2018); Mirzasoleiman\net al. (2018). A streaming algorithm with low adaptive complexity has recently been developed by\nKazemi et al. (2019). In the following, the algorithms are allowed to make an arbitrary number of\npasses through the data.\nCurrently, the best approximation ratio of any algorithm for MCC is 0.385 of Buchbinder and\nFeldman (2016). Their algorithm also works under a more general constraint than cardinality\nconstraint; namely, a matroid constraint. This algorithm is the latest in a series of works (e.g. (Naor\nand Schwartz, 2011; Ene and Nguyen, 2016)) using the multilinear extension of a submodular\nfunction, which is expensive to evaluate.\n\nPreliminaries\nGiven n \u2208 N, the notation [n] is used for the set {0, 1, . . . , n \u2212 1}. In this work, functions f with\ndomain all subsets of a \ufb01nite set are considered; hence, without loss of generality, the domain of\nthe function f is taken to be 2[n], which is all subsets of [n]. An equivalent characterization of\nsubmodularity is that for each A, B \u2286 [n], f (A \u222a B) + f (A \u2229 B) \u2264 f (A) + f (B). For brevity, the\nnotation fx(A) is used to denote the marginal gain f (A \u222a {x}) \u2212 f (A) of adding element x to set A.\nIn the following, the problem studied is to maximize a submodular function under a cardinality\nconstraint (MCC), which is formally de\ufb01ned as follows. Let f : 2n \u2192 R+ be submodular; let k \u2208 [n].\nThen the problem is to determine\n\narg max\nA\u2286[n]:|A|\u2264k\n\nf (A).\n\nAn instance of MCC is the pair (f, k); however, rather than an explicit description of f, the function\nf is accessed by a value oracle; the value oracle may be queried on any set A \u2286 [n] to yield f (A).\nThe ef\ufb01ciency or runtime of an algorithm is measured by the number of queries made to the oracle\nfor f.\nFinally, without loss of generality, instances of MCC considered in the following satisfy n \u2265 4k. If\nthis condition does not hold, the function may be extended to [m] by adding dummy elements to the\ndomain which do not change the function value. That is, the function g : 2m \u2192 R+ is de\ufb01ned as\ng(A) = f (A \u2229 [n]); it may be easily checked that g remains submodular, and any possible solution\nto the MCC instance (g, k) maps2 to a solution of (f, k) of the same value. Hence, the ratio of any\nsolution to (g, k) to the optimal is the same as the ratio of the mapped solution to the optimal on\n(f, k).\n\n2The mapping is to discard all elements greater than n.\n\n3\n\n\f2 Approximation Algorithms\n\nIn this section, the approximation algorithms based upon interlacing greedy procedures are pre-\nsented. In Section 2.1, the technique is demonstrated with standard greedy procedures in algorithm\nInterlaceGreedy. In Section 2.2, the nearly linear-time algorithm FastInterlaceGreedy is introduced.\n\n2.1 The InterlaceGreedy Algorithm\n\nIn this section, the InterlaceGreedy algorithm (InterlaceGreedy, Alg. 1) is introduced. InterlaceGreedy\ntakes as input an instance of MCC and outputs a set C.\n\nai \u2190 arg maxx\u2208[n]\\(Ai\u222aBi) fx(Ai)\nbi \u2190 arg maxx\u2208[n]\\(Ai+1\u222aBi) fx(Bi)\n\nAlgorithm 1 InterlaceGreedy (f, k): The InterlaceGreedy Algorithm\n1: Input: f : 2[n] \u2192 R+, k \u2208 [n]\n2: Output: C \u2286 [n], such that |C| \u2264 k.\n3: A0 \u2190 B0 \u2190 \u2205\n4: for i \u2190 0 to k \u2212 1 do\n5:\n6: Ai+1 \u2190 Ai + ai\n7:\n8: Bi+1 \u2190 Bi + bi\n9: D1 \u2190 E1 \u2190 {a0}\n10: for i \u2190 1 to k \u2212 1 do\ndi \u2190 arg maxx\u2208[n]\\(Di\u222aEi) fx(Di)\n11:\n12: Di+1 \u2190 Di + di\nei \u2190 arg maxx\u2208[n]\\(Di+1\u222aEi) fx(Ei)\n13:\nEi+1 \u2190 Ei + ei\n14:\n15: return C \u2190 arg max{f (Ai), f (Bi), f (Di), f (Ei) : i \u2208 [k + 1]}\n\nInterlaceGreedy operates by interlacing two standard greedy procedures. This interlacing is ac-\ncomplished by maintaining two disjoint sets A and B, which are initially empty. For k iterations,\nthe element a (cid:54)\u2208 B with the highest marginal gain with respect to A is added to A, followed by\nan analogous greedy selection for B; that is, the element b (cid:54)\u2208 A with the highest marginal gain\nwith respect to B is added to B. After the \ufb01rst set of interlaced greedy procedures complete, a\nmodi\ufb01ed version is repeated with sets D, E, which are initialized to the maximum-value singleton\n{a0}. Finally, the algorithm returns the set with the maximum f-value of any query the algorithm\nhas made to f.\nIf f is submodular, InterlaceGreedy has an approximation ratio of 1/4 and query complexity O(kn);\nthe deterministic algorithm of Gupta et al. (2010) has the same time complexity to achieve ratio 1/6.\nThe full proof of Theorem 1 is provided in Appendix A.\nTheorem 1. Let f : 2[n] \u2192 R+ be submodular, let k \u2208 [n], let O = arg max|S|\u2264k f (S), and let\nC = InterlaceGreedy (f, k). Then\n\nf (C) \u2265 f (O)/4,\n\nand InterlaceGreedy makes O(kn) queries to f.\n\nProof sketch. The argument of Fisher et al. (1978) shows that the greedy algorithm is a (1/2)-\napproximation for monotone submodular maximization with respect to a matroid constraint. This\nargument also applies to non-monotone, submodular functions, but it shows only that f (S) \u2265\n2 f (O \u222a S), where S is returned by the greedy algorithm. Since f is non-monotone, it is possible for\nf (O \u222a S) < f (S). The main idea of the InterlaceGreedy algorithm is to exploit the fact that if S and\nT are disjoint,\n\n1\n\nf (O \u222a S) + f (O \u222a T ) \u2265 f (O) + f (O \u222a S \u222a T ) \u2265 f (O),\n\n(1)\nwhich is a consequence of the submodularity of f. Therefore, by interlacing two greedy procedures,\n2 f (O \u222a A) and\ntwo disjoint sets A,B are obtained, which can be shown to almost satisfy f (A) \u2265 1\nf (B) \u2265 1\n2 f (O \u222a B), after which the result follows from (1). There is a technicality wherein the\nelement a0 must be handled separately, which requires the second round of interlacing to address.\n\n4\n\n\f2.2 The FastInterlaceGreedy Algorithm\n\nIn this section, a faster interlaced greedy algorithm (FastInterlaceGreedy (FIG), Alg. 2) is formulated,\nwhich requires O(n log k) queries. As input, an instance (f, k) of MCC is taken, as well as a\nparameter \u03b4 > 0.\n\nAlgorithm 2 FIG (f, k, \u03b4): The FastInterlaceGreedy Algorithm\n1: Input: f : 2[n] \u2192 R+, k \u2208 [n]\n2: Output: C \u2286 [n], such that |C| \u2264 k.\n3: A0 \u2190 B0 \u2190 \u2205\n4: M \u2190 \u03c4A \u2190 \u03c4B \u2190 maxx\u2208[n] f (x)\n5: i \u2190 \u22121, a\u22121 \u2190 0, b\u22121 \u2190 0\n6: while \u03c4A \u2265 \u03b4M/k or \u03c4B \u2265 \u03b4M/k do\n(ai+1, \u03c4A) \u2190 ADD(A, B, ai, \u03c4A)\n7:\n(bi+1, \u03c4B) \u2190 ADD(B, A, bi, \u03c4B)\n8:\ni \u2190 i + 1\n9:\n10: D1 \u2190 E1 \u2190 {a0}, \u03c4D \u2190 \u03c4E \u2190 M\n11: i \u2190 0, d0 \u2190 0, e0 \u2190 0\n12: while \u03c4D \u2265 \u03b4M/k or \u03c4E \u2265 \u03b4M/k do\n13:\n14:\n15:\n16: return C \u2190 arg max{f (A), f (B), f (D), f (E)}\n\n(di+1, \u03c4D) \u2190 ADD(D, E, di, \u03c4D)\n(ei+1, \u03c4E) \u2190 ADD(E, D, ei, \u03c4E)\ni \u2190 i + 1\n\nreturn (0, (1 \u2212 \u03b4)\u03c4 )\nfor (x \u2190 j; x < n; x \u2190 x + 1) do\n\nAlgorithm 3 ADD (S, T, j, \u03c4 ): The ADD subroutine\n1: Input: Two sets S, T \u2286 [n], element j \u2208 [n], \u03c4 \u2208 R+\n2: Output: (i, \u03c4 ), such that i \u2208 [n], \u03c4 \u2208 R+\n3: if |S| = k then\n4:\n5: while \u03c4 \u2265 \u03b4M/k do\n6:\nif x (cid:54)\u2208 T then\n7:\n8:\n9:\n10:\n11:\n12:\n13: return (0, \u03c4 )\n\nif fx(S) \u2265 \u03c4 then\nS \u2190 S \u222a {x}\nreturn (x, \u03c4 )\n\n\u03c4 \u2190 (1 \u2212 \u03b4)\u03c4\nj \u2190 0\n\nThe algorithm FIG works as follows. As in InterlaceGreedy, there is a repeated interlacing of two\ngreedy procedures. However, to ensure a faster query complexity, these greedy procedures are\nthresholded: a separate threshold \u03c4 is maintained for each of the greedy procedures. The interlacing\nis accomplished by alternating calls to the ADD subroutine (Alg. 3), which adds a single element\nand is described below. When all of the thresholds fall below the value \u03b4M/k, the maximum of\nthe greedy solutions is returned; here, \u03b4 > 0 is the input parameter, M is the maximum value of a\nsingleton, and k \u2264 n is the cardinality constraint.\nThe ADD subroutine is responsible for adding a single element above the input threshold and decreasing\nthe threshold. It takes as input four parameters: two sets S, T , element j, and threshold \u03c4; furthermore,\nADD is given access to the oracle f, the budget k, and the parameter \u03b4 of FIG. As an overview, ADD\nadds the \ufb01rst3 element x \u2265 j, such that x (cid:54)\u2208 T and such that the marginal gain fx(S) is at least \u03c4. If\nno such element x \u2265 j exists, the threshold is decreased by a factor of (1 \u2212 \u03b4) and the process is\nrepeated (with j set to 0). When such an element x is found, the element x is added to S, and the new\nthreshold value and position x are returned. Finally, ADD ensures that the size of S does not exceed k.\nNext, the approximation ratio of FIG is proven.\n\n3The \ufb01rst element x > j in the natural ordering on [n] = {0, . . . , n \u2212 1}.\n\n5\n\n\fTheorem 2. Let f : 2[n] \u2192 R+ be submodular, let k \u2208 [n], and let \u03b5 > 0. Let O =\narg max|S|\u2264k f (S). Choose \u03b4 such that (1 \u2212 6\u03b4)/4 > 1/4 \u2212 \u03b5, and let C = FIG (f, k, \u03b4). Then\n\nf (C) \u2265 (1 \u2212 6\u03b4)f (O)/4 \u2265 (1/4 \u2212 \u03b5) f (O).\n\nProof. Let A, B, C, D, E, M have their values at\nLet A =\n{a0, . . . , a|A|\u22121} be ordered by addition of elements by FIG into A. The proof requires the fol-\nlowing four inequalities:\n\ntermination of FIG(f, k, \u03b4).\n\nf (O \u222a A) \u2264 (2 + 2\u03b4)f (A) + \u03b4M,\nf ((O \\ {a0}) \u222a B) \u2264 (2 + 2\u03b4)f (B) + \u03b4M,\nf (O \u222a D) \u2264 (2 + 2\u03b4)f (D) + \u03b4M,\nf (O \u222a E) \u2264 (2 + 2\u03b4)f (E) + \u03b4M.\n\n(2)\n(3)\n(4)\n(5)\nOnce these inequalities have been established, Inequalities 2, 3, submodularity of f, and A \u2229 B = \u2205\nimply\nSimilarly, from Inequalities 4, 5, submodularity of f, and D \u2229 E = {a0}, it holds that\n\nf (O \\ {a0}) \u2264 2(1 + \u03b4)(f (A) + f (B)) + 2\u03b4M.\nf (O \u222a {a0}) \u2264 2(1 + \u03b4)(f (D) + f (E)) + 2\u03b4M.\n\nHence, from the fact that either a0 \u2208 O or a0 (cid:54)\u2208 O and the de\ufb01nition of C, it holds that\n\n(7)\n\n(6)\n\nf (O) \u2264 4(1 + \u03b4)f (C) + 2\u03b4M.\n\nSince f (C) \u2264 f (O) and M \u2264 f (O), the theorem is proved.\nThe proofs of Inequalities 2\u20135 are similar. The proof of Inequality 3 is given here, while the proofs of\nthe others are provided in Appendix B.\nProof of Inequality 3. Let A = {a0, . . . , a|A|\u22121} be ordered as speci\ufb01ed by FIG. Likewise, let\nB = {b0, . . . , b|B|\u22121} be ordered as speci\ufb01ed by FIG.\nLemma 1. O \\ (B \u222a {a0}) = {o0, . . . , ol\u22121} can be ordered such that\n\nfoi(Bi) \u2264 (1 + 2\u03b4)fbi(Bi),\n\n(8)\n\nfor any i \u2208 [|B|].\nProof. For each i \u2208 [|B|], de\ufb01ne \u03c4Bi to be the value of \u03c4 when bi was added into B by the ADD\nsubroutine. Order o \u2208 (O \\ (B \u222a {a0})) \u2229 A = {o0, . . . , o(cid:96)\u22121} by the order in which these elements\nwere added into A. Order the remaining elements of O \\ (B \u222a {a0}) arbitrarily. Then, when bi w;as\nchosen by ADD, it holds that oi (cid:54)\u2208 Ai+1, since A1 = {a0} and a0 (cid:54)\u2208 O \\ (B \u222a {a0}). Also, it holds\nthat oi (cid:54)\u2208 Bi since Bi \u2286 B; hence oi was not added into some (possibly non-proper) subset B(cid:48)\ni of\nBi at the previous threshold value \u03c4Bi\n(1\u2212\u03b4). Since\nfbi(Bi) \u2265 \u03c4Bi and \u03b4 < 1/2, inequality (8) follows.\nOrder \u02c6O = O \\ (B \u222a {a0}) = {o0, . . . , ol\u22121} as de\ufb01ned in the proof of Lemma 1, and let \u02c6Oi =\n{o0, . . . , oi\u22121}, if i \u2265 1, and let \u02c6O0 = \u2205. Then\n\n(1\u2212\u03b4). By submodularity, foi(Bi) \u2264 foi(B(cid:48)\n\ni) < \u03c4Bi\n\n(1 + 2\u03b4)fbi(Bi) +\n\nfoi(B)\n\nf ( \u02c6O \u222a B) \u2212 f (B) =\n\n=\n\n\u2264\n\n\u2264\n\ni=0\n\nl\u22121(cid:88)\nfoi( \u02c6Oi \u222a B)\n|B|\u22121(cid:88)\n|B|\u22121(cid:88)\n|B|\u22121(cid:88)\n\nfoi( \u02c6Oi \u222a B) +\nl\u22121(cid:88)\n\nfoi (Bi) +\n\ni=0\n\ni=0\n\ni=|B|\n\ni=0\n\n\u2264 (1 + 2\u03b4)f (B) + \u03b4M,\n\n6\n\nl\u22121(cid:88)\n\ni=|B|\n\nfoi ( \u02c6Oi \u222a B)\n\nfoi(B)\n\nl\u22121(cid:88)\n\ni=|B|\n\n\fwhere any empty sum is de\ufb01ned to be 0; the \ufb01rst inequality follows by submodularity, the second\nfollows from Lemma 1, and the third follows from the de\ufb01nition of B, and the facts that, for any i\nsuch that |B| \u2264 i < l, maxx\u2208[n]\\A|B|+1 fx(B) < \u03b4M/k, l \u2212 |B| \u2264 k, and oi (cid:54)\u2208 A|B|+1.\nTheorem 3. Let f : 2[n] \u2192 R+ be submodular, let k \u2208 [n], and let \u03b4 > 0. Then the number of\n\nqueries to f by FIG(f, k, \u03b4) is at most O(cid:0) n\n\n(cid:1).\n\n\u03b4 log k\n\n\u03b4\n\nProof. Recall [n] = {0, 1, . . . , n \u2212 1}. Let S \u2208 {A, B, D, E}, and S = {s0, . . . , s|S|\u22121} in the\norder in which elements were added to S. When ADD is called by FIG to add an element si \u2208 [n] to\nS, if the value of \u03c4 is the same as the value when si\u22121 was added to S, then si > si\u22121. Finally, once\nADD queries the marginal gain of adding (n \u2212 1), the threshold is revised downward by a factor of\n(1 \u2212 \u03b4).\nTherefore, there are at most O(n) queries of f at each distinct value of \u03c4A, \u03c4B, \u03c4D, \u03c4E. Since at most\nO( 1\n\n\u03b4 ) values are assumed by each of these thresholds, the theorem follows.\n\n\u03b4 log k\n\n3 Tight Examples\n\nIn this section, examples are provided showing that InterlaceGreedy or FastInterlaceGreedy may\nachieve performance ratio at most 1/4 + \u03b5 on speci\ufb01c instances, for each \u03b5 > 0. These examples\nshow that the analysis in the preceding sections is tight.\nLet \u03b5 > 0 and choose k such that 1/k < \u03b5. Let O and D be disjoint sets each of k distinct elements;\nand let U = O \u02d9\u222a{a, b} \u02d9\u222aD. A submodular function f will be de\ufb01ned on subsets of U as follows.\nLet C \u2286 U.\n\n\u2022 If both a \u2208 C and b \u2208 C, then f (C) = 0.\n|C\u2229O|\n\u2022 If a \u2208 C xor b \u2208 C, then f (C) =\nk .\n2k + 1\n|C\u2229O|\n\u2022 If a (cid:54)\u2208 C and b (cid:54)\u2208 C, then f (C) =\n\n.\n\nk\n\nThe following proposition is proved in Appendix D.\nProposition 1. The function f is submodular.\nNext, observe that for any o \u2208 O, fa(\u2205) = fb(\u2205) = fo(\u2205) = 1/k. Hence InterlaceGreedy or\nFastInterlaceGreedy may choose a0 = a and b0 = b; after this choice, the only way to increase f\nis by choosing elements of O. Hence ai, bi will be chosen in O until elements of O are exhausted,\nwhich results in k/2 elements of O added to each of A and B. Thereafter, elements of D will be\nchosen, which do not affect the function value. This yields\n\nf (A) = f (B) \u2264 1/k + 1/4.\n\nNext, D1 = E1 = {a}, and a similar situation arises, in which k/2 elements of O are added to D, E,\nyielding f (D) = f (E) = f (A). Hence InterlaceGreedy or FastInterlaceGreedy may return A, while\nf (O) = 1. So f (A)\n\nf (O) \u2264 1/k + 1/4 \u2264 1/4 + \u03b5.\n\n4 Experimental Evaluation\n\nIn this section, performance of FastInterlaceGreedy (FIG) is compared with that of state-of-the-art\nalgorithms on two applications of submodular maximization: cardinality-constrained maximum cut\nand network monitoring.\n\n4.1 Setup\n\nAlgorithms The following algorithms are compared.\nated implementations of all algorithms\nnon-monotone-max-cardinality.\n\nthe evalu-\nis available at https://gitlab.com/kuhnle/\n\nSource code for\n\n7\n\n\f\u2022 FastInterlaceGreedy (Alg. 2): FIG is implemented as speci\ufb01ed in the pseudocode, with the\nfollowing addition: a stealing procedure is employed at the end, which uses submodularity\nto quickly steal4 elements from A, B, D, E into C in O(k) queries. This does not impact\nthe performance guarantee, as the value of C can only increase. The parameter \u03b4 is set to\n0.1, yielding approximation ratio of 0.1.\n\n\u2022 Gupta et al. (2010): The algorithm of Gupta et al. (2010) for cardinality constraint; as the\nsubroutine for the unconstrained maximization subproblems, the deterministic, linear-time\n1/3-approximation algorithm of Buchbinder et al. (2012) is employed. This yields an overall\napproximation ratio of 1/7 for the implementation used herein. This algorithm is the fastest\ndetermistic approximation algorithm in prior literature.\n\n(cid:1) randomized algorithm of Buchbinder et al.\n\u2022 FastRandomGreedy (FRG): The O(cid:0) n\n\u2022 BLITS: The O(cid:0)log2 n(cid:1)-adaptive algorithm recently introduced in Balkanski et al. (2018);\n\n(2015) (Alg. 4 of that paper), with expected ratio 1/e \u2212 \u03b5; the parameter \u03b5 was set to\n0.3, yielding expected ratio of \u2248 0.07 as evaluated herein. This algorithm is the fastest\nrandomized approximation algorithm in prior literature.\n\n\u03b52 ln 1\n\n\u03b5\n\nthe algorithm is employed as a heuristic without performance ratio, with the same parameter\nchoices as in Balkanski et al. (2018). In particular, \u03b5 = 0.3 and 30 samples are used to\napproximate the expections. Also, a bound on OPT is guessed in logarithmically many\niterations as described in Balkanski et al. (2018) and references therein.\n\nResults for randomized algorithms are the mean of 10 trials, and the standard deviation is represented\nin plots by a shaded region.\n\nApplications Many applications with non-monotone, submodular objective functions exist. In this\nsection, two applications are chosen to demonstrate the performance of the evaluated algorithms.\n\n\u2022 Cardinality-Constrained Maximum Cut: The archetype of a submodular, non-monotone\nfunction is the maximum cut objective: given graph G = (V, E), S \u2286 V , f (S) is de\ufb01ned to\nbe the number of edges crossing from S to V \\ S. The cardinality constrained version of\nthis problem is considered in the evaluation.\n\n\u2022 Social Network Monitoring: Given an online social network, suppose it is desired to choose\nk users to monitor, such that the maximum amount of content is propagated through these\nusers. Suppose the amount of content propagated between two users u, v is encoded as\n\nweight w(u, v). Then f (S) =(cid:80)\n\nu\u2208S,v(cid:54)\u2208S w(u, v).\n\n4.2 Results\n\nIn this section, results are presented for the algorithms on the two applications. In overview: in terms\nof objective value, FIG and Gupta et al. (2010) were about the same and outperformed BLITS and\nFRG. Meanwhile, FIG was the fastest algorithm by the metric of queries to the objective and was\nfaster than Gupta et al. (2010) by at least an order of magnitude.\n\nCardinality Constrained MaxCut For these experiments, two random graph models were em-\nployed: an Erd\u02ddos-R\u00e9nyi (ER) random graph with 1, 000 nodes and edge probability p = 1/2, and a\nBarab\u00e1si\u2013Albert (BA) graph with n = 10, 000 and m = m0 = 100.\nOn the ER graph, results are shown in Figs. 1(a) and 1(b); the results on the BA graph are shown in\nFigs. 1(c) and 1(d). In terms of cut value, the algorithm of Gupta et al. (2010) performed the best,\nalthough the value produced by FIG was nearly the same. On the ER graph, the next best was FRG\nfollowed by BLITS; whereas on the BA graph, BLITS outperformed FRG in cut value. In terms of\nef\ufb01ciency of queries, FIG used the smallest number on every evaluated instance, although the number\ndid increase logarithmically with budget. The number of queries used by FRG was higher, but after\na certain budget remained constant. The next most ef\ufb01cient was Gupta et al. (2010) followed by\nBLITS.\n\n4Details of the stealing procedure are given in Appendix C.\n\n8\n\n\f(a) ER, Cut Value\n\n(b) ER, Function Queries\n\n(c) BA, Cut Value\n\n(d) BA, Function Queries\n\n(e) Total content monitored versus\nbudget k\n\n(f) Number of Queries versus bud-\nget k\n\nFigure 1: (a)\u2013(d): Objective value and runtime for cardinality-constrained maxcut on random graphs.\n(e)\u2013(f): Objective value and runtime for cardinality-constrained maxcut on ca-AstroPh with simulated\namounts of content between users. In all plots, the x-axis shows the budget k.\n\n(a) ER instance, n = 1000\n\n(b) BA instance, n = 10000\n\nFigure 2: Effect of stealing procedure on solution quality of FIG.\n\nSocial Network Monitoring For the social network monitoring application, the citation network\nca-AstroPh from the SNAP dataset collection was used, with n = 18, 772 users and 198, 110 edges.\nEdge weights, which represent the amount of content shared between users, were generated uniformly\nrandomly in [1, 10]. The results were similar qualitatively to those for the unweighted MaxCut\nproblem presented previously. FIG is the most ef\ufb01cient in terms of number of queries, and FIG is\nonly outperformed in solution quality by Gupta et al. (2010), which required more than an order of\nmagnitude more queries.\n\nEffect of Stealing Procedure\nIn Fig. 2 above, the effect of removing the stealing procedure is\nshown on the random graph instances. Let CF IG be the solution returned by FIG, and CF IG\u2217 be\nthe solution returned by FIG with the stealing procedure removed. Fig. 2(a) shows that on the ER\ninstance, the stealing procedure adds at most 1.5% to the solution value; however, on the BA instance,\nFig. 2(b) shows that the stealing procedure contributes up to 45% increase in solution value, although\nthis effect degrades with larger k. This behavior may be explained by the interlaced greedy process\nbeing forced to leave good elements out of its solution, which are then recovered during the stealing\nprocedure.\n\n9\n\n200400k2.55.07.510.012.5Value x 104FIGBlitsFRGGupta et al.200400k104105106Number of QueriesFIGBlitsFRGGupta et al.10002000k7.510.012.515.0Value x 104FIGBlitsFRGGupta et al.10002000k106107108Number of QueriesFIGBlitsFRGGupta et al.2505007501000k1234Value x 105FIGBlitsFRGGupta et al.2505007501000k106107108Number of QueriesFIGBlitsFRGGupta et al.100200300400k1.0051.0101.015f(CFIG)/f(CFIG*)500100015002000k1.21.31.4f(CFIG)/f(CFIG*)\f5 Acknowledgements\n\nThe work of A. Kuhnle was partially supported by Florida State University and the Informatics\nInstitute of the University of Florida. Victoria G. Crawford and the anonymous reviewers provided\nhelpful feedback which improved the paper.\n\nReferences\nAshwinkumar Badanidiyuru and Jan Vondr\u00e1k. Fast algorithms for maximizing submodular functions.\n\nACM-SIAM Symposium on Discrete Algorithms (SODA), 2014.\n\nEric Balkanski, Adam Breuer, and Yaron Singer. Non-monotone Submodular Maximization in\nExponentially Fewer Iterations. In Advances in Neural Information Processing Systems (NeurIPS),\n2018.\n\nNiv Buchbinder and Moran Feldman. Constrained Submodular Maximization via a Non-symmetric\n\nTechnique. In arXiv preprint arXiv:1611.03253v1, 2016.\n\nNiv Buchbinder and Moran Feldman. Deterministic Algorithms for Submodular Maximization. ACM\n\nTransactions on Algorithms, 14(3), 2018a.\n\nNiv Buchbinder and Moran Feldman. Submodular Functions Maximization Problems \u2013 A Survey. In\nTeo\ufb01lo F. Gonzalez, editor, Handbook of Approximation Algorithms and Metaheuristics. Second\nedition, 2018b.\n\nNiv Buchbinder, Moran Feldman, Joseph Sef\ufb01 Naor, and Roy Schwartz. A Tight Linear Time (1 /\n2)-Approximation for Unconstrained Submodular Maximization. In Symposium on Foundations of\nComputer Science (FOCS), 2012.\n\nNiv Buchbinder, Moran Feldman, Joseph (Sef\ufb01) Naor, and Roy Schwartz. Submodular Maximization\nwith Cardinality Constraints. ACM-SIAM Symposium on Discrete Algorithms (SODA), pages\n1433\u20131452, 2014.\n\nNiv Buchbinder, Moran Feldman, and Roy Schwartz. Comparing Apples and Oranges: Query\nTradeoff in Submodular Maximization. In ACM-SIAM Symposium on Discrete Algorithms (SODA),\n2015.\n\nChandra Chekuri, Shalmoli Gupta, and Kent Quanrud. Streaming Algorithms for Submodular\nFunction Maximization. In International Colloquium on Automata, Languages, and Programming\n(ICALP), 2015.\n\nAlina Ene and Huy L. Nguyen. Constrained Submodular Maximization: Beyond 1/e. In Symposium\n\non Foundations of Computer Science (FOCS), 2016.\n\nAlina Ene and Huy L. Nguyen. Parallel Algorithm for Non-Monotone DR-Submodular Maximization.\n\nIn arXiv preprint arXiv 1905:13272, 2019.\n\nMatthew Fahrbach, Vahab Mirrokni, and Morteza Zadimoghaddam. Non-monotone Submodular\nIn International Conference on\n\nMaximization with Nearly Optimal Adaptivity Complexity.\nMachine Learning (ICML), 2019.\n\nUriel Feige, Vahab S. Mirrokni, and Jan Vondr\u00e1k. Maximizing Non-Monotone Submodular Functions.\n\nSIAM Journal on Computing, 40(4):1133\u20131153, 2011.\n\nMoran Feldman, Christopher Harshaw, and Amin Karbasi. Greed is Good: Near-Optimal Submodular\n\nMaximization via Greedy Optimization. In Conference on Learning Theory (COLT), 2017.\n\nMoran Feldman, Amin Karbasi, and Ehsan Kazemi. Do less, get more: Streaming submodular\nmaximization with subsampling. In Advances in Neural Information Processing Systems (NeurIPS),\n2018.\n\nM.L. Fisher, G.L. Nemhauser, and L.A. Wolsey. An analysis of approximations for maximizing\n\nsubmodular set functions-II. Mathematical Programming, 8:73\u201387, 1978.\n\n10\n\n\fJennifer Gillenwater, Alex Kulesza, and Ben Taskar. Near-Optimal MAP Inference for Determinantal\n\nPoint Processes. In Advances in Neural Information Processing Systems (NeurIPS), 2012.\n\nAnupam Gupta, Aaron Roth, Grant Schoenebeck, and Kunal Talwar. Constrained non-monotone\nsubmodular maximization: Of\ufb02ine and secretary algorithms. In International Workshop on Internet\nand Network Economics (WINE), 2010.\n\nEhsan Kazemi, Marko Mitrovic, Morteza Zadimoghaddam, Silvio Lattanzi, and Amin Karbasi.\nSubmodular Streaming in All its Glory: Tight Approximation, Minimum Memory and Low\nAdaptive Complexity. In International Conference on Machine Learning (ICML), 2019.\n\nDavid Kempe, Jon Kleinberg, and \u00c9va Tardos. Maximizing the spread of in\ufb02uence through a social\nnetwork. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining\n(KDD), 2003.\n\nJon Lee, Vahab Mirrokni, Viswanath Nagarajan, and Maxim Sviridenko. Maximizing Nonmonotone\nSubmodular Functions under Matroid or Knapsack Constraints. Siam Journal of Discrete Math, 23\n(4):2053\u20132078, 2010.\n\nJure Leskovec, Andreas Krause, Carlos Guestrin, Christos Faloutsos, Jeanne VanBriesen, and Na-\ntalie Glance. Cost-effective Outbreak Detection in Networks. In ACM SIGKDD International\nConference on Knowledge Discovery and Data Mining (KDD), 2007.\n\nBaharan Mirzasoleiman, Ashwinkumar Badanidiyuru, Amin Karbasi, Jan Vondrak, and Andreas\nKrause. Lazier Than Lazy Greedy. In AAAI Conference on Arti\ufb01cial Intelligence (AAAI), 2015.\n\nBaharan Mirzasoleiman, Ashwinkumar Badanidiyuru, and Amin Karbasi. Fast Constrained Submod-\nular Maximization : Personalized Data Summarization. In International Conference on Machine\nLearning (ICML), 2016.\n\nBaharan Mirzasoleiman, Stefanie Jegelka, and Andreas Krause. Streaming Non-Monotone Submodu-\nlar Maximization: Personalized Video Summarization on the Fly. In AAAI Conference on Arti\ufb01cial\nIntelligence, 2018.\n\nJoseph Sef\ufb01 Naor and Roy Schwartz. A Uni\ufb01ed Continuous Greedy Algorithm for Submodular\n\nMaximization. In Symposium on Foundations of Computer Science (FOCS), 2011.\n\nG L Nemhauser and L A Wolsey. Best Algorithms for Approximating the Maximum of a Submodular\n\nSet Function. Mathematics of Operations Research, 3(3):177\u2013188, 1978.\n\nG. L. Nemhauser, L. A. Wolsey, and M. L. Fisher. An analysis of approximations for maximizing\n\nsubmodular set functions-I. Mathematical Programming, 14(1):265\u2013294, 1978.\n\nJan Vondr\u00e1k. Symmetry and Approximability of Submodular Maximization Problems. SIAM Journal\n\non Computing, 42(1):265\u2013304, 2013.\n\n11\n\n\f", "award": [], "sourceid": 1393, "authors": [{"given_name": "Alan", "family_name": "Kuhnle", "institution": "Florida State University"}]}