{"title": "Optimistic optimization of a Brownian", "book": "Advances in Neural Information Processing Systems", "page_first": 3005, "page_last": 3014, "abstract": "We address the problem of optimizing a Brownian motion. We consider a (random) realization $W$ of a Brownian motion  with input space in $[0,1]$. Given $W$, our goal is to return an $\\epsilon$-approximation of its maximum using the smallest possible number of function evaluations, the sample complexity of the algorithm. We provide an algorithm with sample complexity of order $\\log^2(1/\\epsilon)$. This improves over previous results of Al-Mharmah and Calvin (1996) and Calvin et al. (2017) which provided only polynomial rates. Our algorithm is adaptive---each query depends on previous values---and is an instance of the  optimism-in-the-face-of-uncertainty principle.", "full_text": "Optimistic Optimization of a Brownian\n\nJean-Bastien Grill\n\nMichal Valko\n\nR\u00b4emi Munos\n\nSequeL team, INRIA Lille - Nord Europe, France and DeepMind Paris, France\n\njbgrill@google.com\n\nmichal.valko@inria.fr\n\nmunos@google.com\n\nAbstract\n\nWe address the problem of optimizing a Brownian motion. We consider a (random)\nrealization W of a Brownian motion with input space in [0, 1]. Given W , our goal\nis to return an \u03b5-approximation of its maximum using the smallest possible number\nof function evaluations, the sample complexity of the algorithm. We provide an\nalgorithm with sample complexity of order log2(1/\u03b5). This improves over previous\nresults of Al-Mharmah and Calvin (1996) and Calvin et al. (2017) which provided\nonly polynomial rates. Our algorithm is adaptive\u2014each query depends on previous\nvalues\u2014and is an instance of the optimism-in-the-face-of-uncertainty principle.\n\n1\n\nIntroduction to sample-ef\ufb01cient Brownian optimization\n\nWe are interested in optimizing a sample of a standard Brownian motion on [0, 1], denoted by W.\nMore precisely, we want to sequentially select query points tn \u2208 [0, 1], observe W (tn), and decide\n\nwhen to stop to return a point(cid:98)t and its value(cid:99)M = W(cid:0)(cid:98)t(cid:1) in order to well approximate its maximum\nas early as possible while still being able to return(cid:98)t such that with probability at least 1 \u2212 \u03b5,\nM \u2212 W(cid:0)(cid:98)t(cid:1) \u2264 \u03b5. The number of function evaluations required by the algorithm to achieve this\n\nM (cid:44) supt\u2208[0,1] W (t). The evaluations tn can be chosen adaptively as a function of previously\nobserved values W (t1), ..., W (tn\u22121). Given an \u03b5 > 0, our goal is to stop evaluating the function\n\n\u03b5-approximation of the maximum de\ufb01nes the sample-complexity.\nMotivation There are two types of situations where this problem is useful. The \ufb01rst type is when the\nrandom sample function W (drawn from the random process) already exists prior to the optimization.\nEither it has been generated before the optimization starts and the queries correspond to reading\nvalues of the function already stored somewhere. For example, \ufb01nancial stocks are stored at a high\ntemporal resolution and we want to retrieve the maximum of a stock using a small number of memory\nqueries. Alternatively, the process physically exists and the queries correspond to measuring it.\nAnother situation is when the function does not exist prior to the optimization but is built simultane-\nously as it is optimized. In other words, observing the function actually creates it. An application of\nthis is when we want to return a sample of the maximum (and the location of the maximum) of a\nBrownian motion conditioned on a set of already observed values. For example, in Bayesian optimiza-\ntion for Gaussian processes, a technique called Thomson sampling (Thompson, 1933; Chapelle and\nLi, 2011; Russo et al., 2018; Basu and Ghosh, 2018) requires returning the maximum of a sampled\nfunction drawn from the posterior distribution. The problem considered in the present paper can\nbe seen as a way to approximately perform this step in a computationally ef\ufb01cient way when this\nGaussian process is a Brownian motion.\nMoreover, even though our algorithm comes from the ideas of learning theory, it has applications\nbeyond it. For instance, in order to computationally sample a solution of a stochastic differential\nequation, Hefter and Herzwurm (2017) express its solution as a function of the Brownian motion W\nand its running minimum. They then need, as a subroutine, an algorithm for the optimization of\nBrownian motion to compute its running minimum. We are giving them that and it is light-speed fast.\n\n32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montr\u00b4eal, Canada.\n\n\fPrior work Al-Mharmah and Calvin (1996) provide a non-adaptive method to optimize a Brownian\nmotion. They prove that their method is optimal among all non-adaptive methods and their sample\n\u03b5. More recently, Calvin et al. (2017) provided an adaptive\ncomplexity is polynomial of order 1/\nalgorithm with a sample complexity lower than any polynomial rate showing that adaptability to\nprevious samples yields a signi\ufb01cant algorithmic improvement. Yet their result does not guarantee a\nbetter rate than a polynomial one.\n\n\u221a\n\nOur contribution We introduce the algorithm OOB = optimistic optimization of the Brownian\nmotion. It uses the optimism-in-face-of-uncertainty apparatus: Given n \u2212 1 points already eval-\nuated, we de\ufb01ne a set of functions Un in which W lies with high probability. We then select\nthe next query point tn where the maximum of the most optimistic function of Un is reached:\ntn (cid:44) arg maxt\u2208[0,1] maxf\u2208Un f (t). This begets a simple algorithm that requires an expected number\nof queries of the order of log2(1/\u03b5) to return an \u03b5-approximation of the maximum, with probability\nat least 1 \u2212 \u03b5 w.r.t. the random sample of the Brownian motion. Therefore, our sample complexity is\nbetter than any polynomial rate.\n\nSolving an open problem Munos (2011) provided sample complexity results for optimizing any\nfunction f characterized by two coef\ufb01cients (d, C) where d is the near-optimality dimension and C\nthe corresponding constant (see his De\ufb01nition 3.1). It is de\ufb01ned as the smallest d \u2265 0 such that\nthere exists a semi-metric (cid:96) and a constant C > 0, such that, for all \u03b5 > 0, the maximal number of\ndisjoint (cid:96)-balls of radius O(\u03b5) with center in {x, f (x) \u2265 supx f (x)\u2212 \u03b5} is less than C\u03b5\u2212d. Under the\nassumption that f is locally (around one global maximum) one-sided Lipschitz with respect to (cid:96) (see\nhis Assumption 2), he proved that for a function f characterized by (d = 0, C), his DOO algorithm\nhas a sample complexity of O(C log(1/\u03b5)), whereas for a function characterized by (d > 0, C),\n\nthe sample complexity of DOO is O(cid:0)C/\u03b5d(cid:1). Our result answers a question he raised: What is the\n\nnear-optimality dimension of a Brownian-motion? The Brownian motion being a stochastic process,\nthis quantity is a random variable so we consider the number of disjoint balls in expectation. We show\nthat for any \u03b5, there exists some particular metric (cid:96)\u03b5 such that the Brownian motion W is (cid:96)\u03b5-Lipschitz\nwith probability 1 \u2212 \u03b5, and there exists a constant C(\u03b5) = O(log(1/\u03b5)) such that (d = 0, C(\u03b5))\ncharacterizes the Brownian motion. However, there exists no constant C < \u221e independent of \u03b5 such\nthat (d = 0, C) characterizes the Brownian motion. Therefore, we solved this open problem. Our\nanswer is compatible with our result that our algorithm has a sample complexity of O(log2(1/\u03b5)).\n\n2 New algorithm for Brownian optimization\n\nOur algorithm OOB is a version of DOO (Munos, 2011) with a modi\ufb01ed upper bound on the function,\nin order to be able to optimize stochastic processes. Consider the points t1 < t1 < ... < tn evaluated\nso far and t0 = 0. OOB de\ufb01nes an upper con\ufb01dence bound B[ti,ti+1] for each interval [ti, ti+1] with\ni \u2208 {0, ..., n \u2212 1} and samples W in the middle of the interval with the highest upper-con\ufb01dence\nbound. Algorithm 1 reveals its pseudo-code.\n\nAlgorithm 1 OOB algorithm\n1: Input: \u03b5\n2: Init: I \u2190 {[0, 1]}, t1 = W (1)\n3: for i = 2, 3, 4, . . . do\n4:\n5:\n6:\n7:\n8:\n9:\n10: end for\n\n[a, b] \u2208 arg maxI\u2208I BI {break ties arbitrarily}\nif \u03b7\u03b5(b \u2212 a) \u2264 \u03b5 then\nend if\nI \u2190 {I \u222a [a, a+b\n\nti \u2190 W(cid:0) a+b\n\nbreak\n\n(cid:1)\n\n2\n\n11: Output: location(cid:98)t\u03b5 \u2190 arg maxti W (ti) and its value W(cid:0)(cid:98)t\u03b5\n(cid:1)\n\n2 , b]}\\{[a, b]}\n\n2 ] \u222a [ a+b\n\n2\n\n\fB[a,b] (cid:44) max(W (a), W (b)) + \u03b7\u03b5(b \u2212 a), where \u2200\u03b4 > 0 s.t. \u03b5\u03b4 \u2264 1\n2\n\nMore formally, let \u03b5 be the required precision, the only given argument of the algorithm. For any\n(cid:115)\n0 \u2264 a < b \u2264 1, the interval [a, b] is associated with an upper bound B[a,b] de\ufb01ned by\n\n(cid:19)\nOOB keeps track of a set I of intervals [a, b] with W already being sampled at a and b. The algorithm\n\ufb01rst samples W (1), W (1) \u223c N (0, 1), in order to initialize the set I to the singleton {[0, 1]}. Then,\nOOB keeps splitting the interval I \u2208 I associated with the highest upper bound BI quam necessarium.\n\n(cid:18) 2\n\n\u03b7\u03b5(\u03b4) (cid:44)\n\n5\u03b4\n2\n\n\u03b5\u03b4\n\nln\n\n\u00b7\n\n,\n\nP(cid:2)M \u2212 W(cid:0)(cid:98)t\u03b5\n\n(cid:1) > \u03b5(cid:3) \u2264 \u03b5\n\n3 Guarantees: OOB is correct and sample-ef\ufb01cient\n\nLet M (cid:44) supt\u2208[0,1] W (t) be the maximum of the Brownian motion,(cid:98)t\u03b5 the output of OOB called with\n\nparameter \u03b5 > 0, and N\u03b5 the number of Brownian evaluations performed until OOB terminates. All\nare random variables that depend on the Brownian motion W . We now voice our main result.\nTheorem 1. There exists a constant c > 0 such that for all \u03b5 < 1/2,\n\nand E[N\u03b5] \u2264 c log2(1/\u03b5).\n\nwhich is at most \u03b5-away from the true maximum. Such guarantee is called probably approximately\ncorrect (PAC). The second inequality quanti\ufb01es performance. We claim that the expectation (over W )\n\nof the Brownian motion, our OOB is deterministic. The only source of randomness comes from the\nrealization of the Brownian. Therefore, being correct means that among all possible realizations of\n\nThe \ufb01rst inequality quanti\ufb01es the correctness of our estimator (cid:99)M\u03b5 = W(cid:0)(cid:98)t\u03b5\n(cid:1). Given a realization\nthe Brownian motion, there is a subset of measure at least 1\u2212 \u03b5 on which OOB outputs an estimate(cid:99)M\u03b5\nof the number of samples that OOB needs to optimize this realization with precision \u03b5 is O(cid:0)log2(1/\u03b5)(cid:1).\nmin(\u03b4, \u03b5) and apply Theorem 1 for \u03b5(cid:48) from which we get P(cid:2)M \u2212 W(cid:0)(cid:98)t\u03b5\n(cid:1) > \u03b5(cid:48)(cid:3) \u2264 \u03b5(cid:48) which is stronger\n(cid:1) > \u03b5(cid:3) \u2264 \u03b4. Similarly, E[N\u03b5(cid:48)] \u2264 c log2(1/\u03b5(cid:48)) \u2264 4c(log(1/\u03b5) + log(1/\u03b4))2.\nthan P(cid:2)M \u2212 W(cid:0)(cid:98)t\u03b5\n(cid:1) > \u03b5(cid:12)(cid:12)W (t1), ..., W (tN\u03b5 )(cid:3) \u2264 \u03b5,\nunfavorable cases, i.e., the Brownian realizations for which(cid:12)(cid:12)M \u2212(cid:99)M\u03b5\n\nRemark 1. Our PAC guarantee is actually stronger than stated in Theorem 1. Indeed, the PAC\nguarantee analysis can be done conditioned on the collected function evaluations and get\n\nCorollary 1. We get the classic (\u03b4, \u03b5)-PAC guarantee easily. For any \u03b4 > 0 and \u03b5 > 0, choose \u03b5(cid:48) =\n\n(cid:12)(cid:12) > \u03b5, are not concentrated on\n\nfrom which taking the expectation on both sides gives the \ufb01rst part of Theorem 1. This means that the\n\nP(cid:2)M \u2212 W(cid:0)(cid:98)t\u03b5\n\nsome subsets of Brownian realizations matching some evaluations in t1, ..., tN\u03b5. In other words, the\nPAC guarantee also holds when restricted to the Brownian realizations matching the evaluations in\nt1, ..., tN\u03b5 only. This is possible because N\u03b5 is not \ufb01xed but depends on the evaluations done by OOB.\nOne difference from the result of Calvin et al. (2017) is that theirs is with respect to the Lp norm. For\n\ntheir algorithm, they prove that with n samples it returns(cid:98)tn \u2208 [0, 1] such that\n\u2203cr,p, E(cid:104)(cid:12)(cid:12)M \u2212 W ((cid:98)tn)(cid:12)(cid:12)p(cid:105)1/p \u2264 cr,p/nr.\n\n\u2200r > 1, p > 1,\n\nTo express their result in the same formalism as ours, we \ufb01rst choose to achieve accuracy \u03b52 and\ncompute the number of samples n\u03b52 needed to achieve it. Then, for p = 1, we apply Markov\ninequality and get that for all r > 1 there exists cr,1 such that\n\nP(cid:2)M \u2212 W ((cid:98)tn\u03b52 ) > \u03b5(cid:3) \u2264 \u03b5\n\nand N\u03b5 \u2264 cr,1/\u03b51/r.\n\nOn the other hand, in our Theorem 1 we give a poly-logarithmic bound for the sample complexity\nand we are in the business because this is better than any polynomial rate.\n\n4 Analysis and the proof of the main theorem\nWe provide a proof of the main result. Let I\ufb01n be the set I of intervals tracked by OOB when it \ufb01nishes.\nWe de\ufb01ne an event C such that for any interval I of the form I = [k/2h, (k + 1)/2h] with k and h\nbeing two integers where 0 \u2264 k < 2h, the process W is lower than BI on the interval I.\n\n3\n\n\f(cid:40)\nDe\ufb01nition 1. Event C is de\ufb01ned as\n2h\u22121(cid:92)\n\n\u221e(cid:92)\n\nC (cid:44)\n\nW (t) \u2264 B[k/2h,(k+1)/2h]\n\n(cid:41)\n\n\u00b7\n\nsup\n\nt\u2208[k/2h,(k+1)/2h]\n\nk=0\n\nh=0\n\nEvent C is a proxy for the Lipschitz condition on W for the pseudo-distance d(x, y) =\n\n(cid:112)|y \u2212 x| ln(1/|y \u2212 x|) because \u03b7\u03b5(\u03b4) scales with \u03b4 as (cid:112)\u03b4 ln (1/\u03b4). We show that it holds\n\nwith high probability. To show it, we make use of the Brownian bridge which is the process\nBr(t) (cid:44) (W (t)|W (1) = 0). Lemma 1 follows from the Markov property of the Brownian combined\nwith a bound on the law of the maximum of Br(t) to bound the probability P[supt\u2208I W (t) \u2265 BI ] for\nany I of the form [k/2h, (k + 1)/2h] and a union bound over all these intervals.\nLemma 1. For any \u03b5, event C from De\ufb01nition 1 holds with high probability. In particular,\n\nP[Cc] \u2264 \u03b55.\n\nProof. For any interval I,\n\nBI = max(W (a), W (b)) + \u03b7\u03b5(b \u2212 a)\n\n=\n\nW (a) + W (b)\n\n2\n\n\u2265 W (a) + W (b)\n\n2\n\n+\n\n+\n\n|W (a) \u2212 W (b)|\n\n(cid:115)(cid:18) W (a) \u2212 W (b)\n\n2\n\n(cid:19)2\n\n+ \u03b7\u03b5(b \u2212 a)\n\n+ (\u03b7\u03b5(b \u2212 a))2\n\n2\n\n(by de\ufb01nition of BI )\n(max(x, y) = (x+y+|x\u2212y|)/2)\n\n(cid:0)\u2200x, y > 0, (x+y)2 \u2265 x2 +y2(cid:1).\n\nWe now de\ufb01ne\n\nand\n\nth,k (cid:44) k\n2h\n\n,\n\nAh,k (cid:44)\n\n\u2206h,k (cid:44) W (th,k) \u2212 W (th,k+1)\n(cid:40)\n\n2\n\n,\n\n\u03b7h (cid:44) \u03b7\u03b5(b \u2212 a),\n\n\u00b7\n\n(cid:41)\n(cid:18)\n(cid:19)\n\u2212 2(x\u2212Wa)(x\u2212Wb)\n\n\u00b7\n\nsup\n\nt\u2208[k/2h,(k+1)/2h]\n\nW (t) > B[k/2h,(k+1)/2h]\n\n(cid:34)\n\n(cid:35)\n\n= exp\n\nb \u2212 a\n\nW (t) > x\n\nsup\nt\u2208[a,b]\n\nP(cid:2)Ah,k\n\nFirst, for any a < b, the law of the maximum of a Brownian bridge gives us\n\n\u2200x\u2265 max(W (a), W (b)) : P\nCombining it with the de\ufb01nition of Ah,k and the \ufb01rst inequality of the proof we get\n(cid:113)\n\n(cid:12)(cid:12)(cid:12)(cid:12)W (a) = Wa, W (b) = Wb\n(cid:12)(cid:12)W (th, k), W (th, k + 1)(cid:3)\n(cid:16) W (th,k+1)\u2212W (th,k)\n(cid:113)\n\uf8eb\uf8ed\u2212 2\n(cid:16)\u22122h+1(cid:16)(cid:113)\n(cid:17)(cid:16)(cid:113)\n(cid:18)\n(cid:18) 2h\n(cid:17)\n(cid:16)\u22122h+1\u03b72\n(cid:83)2h\u22121\nh,k =(cid:83)\u221e\n(cid:84)2h\u22121\nBy de\ufb01nition, C (cid:44)(cid:84)\u221e\nk=0 Ah,k. By union bound on all Ah,k we get\nk=0 Ac\nP[Cc] \u2264(cid:88)\nP[Ah,k] \u2264(cid:88)\n2h\u22121(cid:88)\n2h\u22121(cid:88)\n\n(cid:17)(cid:17)\n(cid:17)5\u00b7\n(cid:17)5 \u2264(cid:88)\n\n(cid:17)(cid:16) W (th,k)\u2212W (th,k+1)\n\nh \u2212 \u2206h,k\n\u22122h+1\n5\n\nth,k+1 \u2212 th,k\n\n2 \u00b7 2h ln\n\n(cid:16) \u03b5\n\n(cid:16) \u03b5\n\n(cid:19)(cid:19)\n\nh,k + \u03b72\n\nh + \u2206h,k\n\n\u22062\n\nh,k + \u03b72\nh\n\n\u22062\n\nh,k + \u03b72\n\n\u22062\n\nh,k + \u03b72\nh\n\n\uf8f6\uf8f8\n\n\u2264 exp\n\n= exp\n\nh\n\n=\n\n2h\n\n= exp\n\n= exp\n\n(cid:17)\n\n2\n\n+\n\n2\n\n+\n\n\u22062\n\n\u03b5\n\nh=0\n\nh=0\n\n\u03b55\n\n24h \u2264 \u03b55.\n\nh\u22651\n\nk=0\n\nh\u22651\n\nk=0\n\n2h\n\nh\u22651\n\nLemma 1 is useful for two reasons. As we bound the sample complexity on event C and the\ncomplementary event in two different ways, we can use Lemma 1 to combine the two bounds to\nprove Proposition 2 in the end. We also use a weak version of it, bounding \u03b55 by \u03b5 to prove our PAC\nguarantee. For this purpose, we combine the de\ufb01nition of C with the termination condition of OOB to\nSince C holds with high probability, we have the following PAC guarantee which is the \ufb01rst part of\nthe main theorem.\n\nget that under event C, the best point(cid:99)M\u03b5 so far, is close to the maximum M of the Brownian up to \u03b5.\n\n4\n\n\fProposition 1. The estimator(cid:99)M\u03b5 = W(cid:0)(cid:98)t\u03b5\n(cid:1) is probably approximately correct with\nP(cid:104)\nM \u2212(cid:99)M\u03b5 > \u03b5\n\n(cid:105) \u2264 \u03b5.\n\nProof. Let Inext = [a, b] be the interval that the algorithm would split next, if it was not\nterminated. Since the algorithm only splits the interval with the highest upper bound then\nBnext = supI\u2208I\ufb01n BI. Also let Imax \u2208 I\ufb01n be one of the intervals where a maximum is reached,\ntmax \u2208 arg maxt\u2208[0,1] W (t) (cid:44) M and tmax \u2208 Imax. Then, on event C,\n\nM \u2264 BImax \u2264 BInext = max(W (a), W (b)) + \u03b7\u03b5(b \u2212 a).\nSince the algorithm terminated, we have that \u03b7\u03b5(b \u2212 a) \u2264 \u03b5 and therefore,\n\nwhich combined with Lemma 1 \ufb01nishes the proof as \u03b55 \u2264 \u03b5.\n\nmax(W (a), W (b)) \u2265 M \u2212 \u03b5,\n\nNh(\u03b7) (cid:44)\n\n(cid:18) k\n\n(cid:19)\n\nIn fact, Proposition 1 is the easy-to-obtain part of the main theorem. We are now left to prove that\nthe number of samples needed to achieve this PAC guarantee is low. As the next step, we de\ufb01ne the\nnear-optimality property. A point t is said to be \u03b7-near-optimal when its value W (t) is close to the\nmaximum M of the Brownian motion up to \u03b7. Check out the precise de\ufb01nition below.\n\n(cid:1) \u2265 M\u2212\u03b7, we say that the point t = (k/2h) is \u03b7-near-\nDe\ufb01nition 2. When an (h, k, \u03b7) veri\ufb01es W(cid:0) k\noptimal. We de\ufb01ne Nh(\u03b7) as the number of \u03b7-near-optimal points among(cid:8)0/2h, 1/2h, ..., 2h/2h(cid:9),\n\n2h\n\nk \u2208 0, ..., 2h, such that W\n\n(cid:12)(cid:12)(cid:12)(cid:12)(cid:26)\n(cid:0)1/2h(cid:1)-near-optimal. Since we use the principle of optimism in face\n\nNotice that Nh(\u03b7) is a random variable that depends on a particular realization of the Brownian\nmotion. The reason why we are interested in the number of near-optimal points is that the points the\nalgorithm will sample are \u03b7\u03b5\nof uncertainty, we consider the upper bound of the Brownian motion and sample where this upper\nbound is the largest. If our bounds on W hold, i.e., under event C, then any interval I with optimistic\nbound BI < M is never split by the algorithm. This is true when C holds because if the maximum\nof W is reached in Imax, then BImax \u2265 M > BI which shows that Imax is always chosen over I.\nTherefore, a necessary condition for an interval [a, b] to be split is that max(W (a), W (b)) \u2265 M \u2212 \u03b7\nwhich means that either a or b or both are \u03b7-near-optimal which is the key point of Lemma 2.\nLemma 2. Under event C, the number of evaluated points N\u03b5 by the algorithm veri\ufb01es\n\n(cid:27)(cid:12)(cid:12)(cid:12)(cid:12)\u00b7\n\n\u2265 M \u2212 \u03b7\n\n2h\n\n(cid:16)\n\n(cid:16)\n\n1/2h(cid:17)(cid:17)\n\nhmax(cid:88)\n\nh=0\n\nN\u03b5 \u2264 2\n\nNh\n\n\u03b7\u03b5\n\n, with hmax being the smallest h such that \u03b7\u03b5\n\n(cid:16)\n1/2h(cid:17) \u2264 \u03b5.\n\nLemma 2 explicitly links the near-optimality from De\ufb01nition 2 with the number of samples N\u03b5\nperformed by OOB before it terminates. Here, we use the optimism-in-face-of-uncertainty principle\nwhich can be applied to any function. In particular, we de\ufb01ne a high-probability event C under which\nthe number of samples is bounded by the number of near-optimal points Nh(\u03b7h) for all h \u2264 hmax.\nProof. Let I = [a, b] be an interval of It such that max(W (a), W (b)) + \u03b7\u03b5(b \u2212 a) < M. Let Inext \u2208\nIt be the interval that the algorithm would split after t function evaluations.. Since the algorithm\nonly splits the interval with the highest upper bound, then BInext = supI\u2208It BI. Moreover, if we let\nImax \u2208 It be one of the intervals where a maximum is reached, tmax \u2208 arg maxt\u2208[0,1] W (t) (cid:44) M\nand tmax \u2208 Imax, then on event C,\n\nmax(W (a), W (b)) + \u03b7\u03b5(b \u2212 a) (cid:44) BI < M \u2264 BImax \u2264 BInext .\n\nTherefore, under C, a necessary condition for an interval I = [a, b] to be split during the execution\nof OOB is that max(W (a), W (b)) \u2265 M \u2212 \u03b7\u03b5(b \u2212 a), which means that either a or b or both are\n\u03b7\u03b5(b \u2212 a)-near-optimal. From the termination condition of the algorithm, we know that any interval\nthat is satisfying I = [k/2h, (k + 1)/2h] with h \u2265 hmax will not be split during the execution.1\nTherefore, another necessary condition for an interval I = [a, b] to be split during the execution is\nthat b \u2212 a > 1/2hmax. Writing \u03b7h (cid:44) \u03b7\u03b5\n\n(cid:0)1/2h(cid:1), we deduce from these two necessary conditions that\n\n1This holds despite \u03b7\u03b5(\u00b7) is not always decreasing.\n\n5\n\n\fh=0\n\n2h\u22121(cid:88)\nN \u2264 hmax(cid:88)\n2h\u22121(cid:88)\n\u2264 hmax(cid:88)\n2h(cid:88)\nhmax(cid:88)\n\nh=0\n\nk=0\n\nk=0\n\n\u2264 2\n\n1\n\n1\n\nh=0\n\nk=0\n\n2h\n\n(cid:26) k\n2h or k + 1\n(cid:26) k\n(cid:26) k\n\nis \u03b7h-near-optimal\n\n(cid:27)\n(cid:26) k + 1\nhmax(cid:88)\n\n2h\n\nh=0\n\n(cid:27)\n(cid:27)\n\n2h is \u03b7h-near-optimal\n\n+ 1\n\n1\n\n2h is \u03b7h-near-optimal\n\n= 2\n\nNh(\u03b7h).\n\n(cid:27)\n\nis \u03b7h-near-optimal\n\nWe now prove a property speci\ufb01c to W by bounding the number of near-optimal points of the\nBrownian motion in expectation. We do it by rewriting it as two Brownian meanders (Durrett et al.,\n1977), both starting at the maximum of the Brownian, one going backward and the other one forward\nwith the Brownian meander W + de\ufb01ned as\n\n\u2200t \u2208 [0, 1] W +(t) (cid:44) |W (\u03c4 + t(1 \u2212 \u03c4 ))|\n\n\u221a\n\n1 \u2212 \u03c4\n\n, where \u03c4 (cid:44) sup{t \u2208 [0, 1] : W (t) = 0}.\n\nWe use that the Brownian meander W + can be seen as a Brownian motion conditioned to be positive\n(\u0414\u0435\u043d\u0438\u0441\u043e\u0432, 1983). This is the main ingredient of Lemma 3.\nLemma 3. For any h and \u03b7, the expected number of near-optimal points is bounded as\n\nE[Nh(\u03b7)] \u2264 6\u03b722h.\n\nThis lemma answers a question raised by Munos (2011): What is the near-optimality dimension\nof the Brownian motion? We set \u03b7h (cid:44) \u03b7\u03b5\n(cid:96)(x, y) = \u03b7\u03b5(|y \u2212 x|), the near-optimality dimension measures the rate of increase of Nh(\u03b7h),\nthe number of \u03b7h-near-optimal points in [0, 1] of the form k/2h. In Lemma 3, we prove that in\n\n(cid:0)1/2h(cid:1). In dimension one with the pseudo-distance\nh2h(cid:1) = O(log(1/\u03b5)), which is constant with respect to h.\n\nexpectation, this number increases as O(cid:0)\u03b72\n\nThis means that for a given \u03b5, there is a metric under which the Brownian is Lipschitz with probability\nat least 1 \u2212 \u03b5 and has a near-optimality dimension d = 0 with C = O(log(1/\u03b5)).\nThe \ufb01nal sample complexity bound is essentially constituted by one O(log(1/\u03b5)) term coming from\nthe standard DOO error for deterministic function optimization and another O(log(1/\u03b5)) term because\nwe need to adapt our pseudo-distance (cid:96) to \u03b5 such that the Brownian is (cid:96)-Lipschitz with probability\n\n1 \u2212 \u03b5. The product of the two gives the \ufb01nal sample complexity bound of O(cid:0)log2(1/\u03b5)(cid:1).\n\nProof of Lemma 3. We denote by W, the Brownian motion whose maximum M is \ufb01rst hit at the\npoint de\ufb01ned as tM = inf{t \u2208 [0, 1]; W (t) = M} and B+ a Brownian meander (Durrett et al., 1977).\nWe also de\ufb01ne\n\n0 (t) (cid:44) M \u2212 W (tM \u2212 t \u00b7 tM ))\n\nB+\n\nand B+\n\n1 (t) (cid:44) M \u2212 W (tM + t(1 \u2212 tM ))\n\n\u221a\n\n\u00b7\n\n\u221a\ntM\n\n1 \u2212 tM\n\nL\n= denotes the equality in distribution, then Theorem 1 of \u0414\u0435\u043d\u0438\u0441\u043e\u0432 (1983) asserts that\n\nIf\n\nWe upper-bound the expected number of \u03b7-near-optimal points for any integer h \u2265 0 and any \u03b7 > 0,\n\nB+ L\n\n= B+\n0\n\nL\n= B+\n\n1 and tM is independent from both B+\n\n0 and B+\n1 .\n\nW\n\n1\n\n\uf8ee\uf8f0 2h(cid:88)\n(cid:26)\n(cid:19)\n(cid:18) k\n(cid:26)(cid:26)\n(cid:18)\n(cid:26)\n\n2h\n\nk=0\n\nW\n\n(cid:18) k\n\n(cid:19)\n\n2h\n\n(cid:19)\n\nE[Nh(\u03b7)] = E\n\n1\n\n(cid:20)\n(cid:20)\n\nE\n\nE\n\n2h(cid:88)\n2h(cid:88)\n\nk=0\n\nk=0\n\n=\n\n=\n\n> M \u2212 \u03b7\n\n> M \u2212 \u03b7 \u2229 k\n\n2h \u2264 tM\n\n1\n\nB+\n0\n\n1\u2212 k\ntM 2h\n\n<\n\n\u03b7\u221a\ntM\n\n2h \u2264 tM\n\u2229 k\n\n(cid:27)\uf8f9\uf8fb =\n(cid:27)\n\n(cid:19)\n\nk=0\n\n2h(cid:88)\n(cid:26)\n(cid:27)(cid:21)\n\n\u222a\n\nE\n\nW\n\nW\n\n2h\n\n(cid:18) k\n(cid:19)\n(cid:20)\n\n(cid:26)\n\n1\n\n(cid:20)\n(cid:26)\n(cid:18) k\n2h(cid:88)\n\n2h\n\nE\n\n+\n\nk=0\n\n1\n\nB+\n1\n\n(cid:27)(cid:21)\n\n> M \u2212 \u03b7\n\n(cid:27)(cid:27)(cid:21)\n\n> M \u2212 \u03b7 \u2229 k\n\n2h > tM\n\n(cid:18) k/2h\u2212tM\n\n(cid:19)\n\n1 \u2212 tM\n\n(cid:27)(cid:21)\n\n.\n\n<\n\n\u03b7\u221a\n1\u2212tM\n\n\u2229 k\n\n2h > tM\n\nIf X and Y are independent then for any function f,\n\nE[f (X, Y )] = E[E[f (X, Y )|Y ]]\n\n\u2264 supy E[f (X, y)|Y = y]\n= supy E[f (X, y)].\n\n(law of total expectation)\n(for any Z, E[Z] \u2264 supw Z(w))\n(because X and Y are independent)\n\n6\n\n\fSince tM is independent from B+\n\n0 and B+\n\n1 , then using the above with X = (B+\n\n0 , B+\n\n1 ), Y = tM , and\n\n(cid:18)\n\n(cid:26)\n\n(cid:18)\n\n1\n\nx0\n\n(cid:19)\n\n1\u2212 k\ntM 2h\n\n<\n\n\u03b7\u221a\ny\n\n\u2229 k\n2h \u2264 y\n\n(cid:27)\n\n(cid:26)\n\n+1\n\nx1\n\n(cid:18) k/2h\u2212tM\n\n(cid:19)\n\n1\u2212y\n\n<\n\n\u03b7\u221a\n1\u2212y\n\n\u2229 k\n\n2h > y\n\n(cid:27)(cid:19)\n\n,\n\nwe can claim that\nE[Nh(\u03b7)] = E[f (X, Y )] \u2264 sup\nu\u2208[0,1]\n\n\u2229 k\n\n2h > u\n\n\u03b7\u221a\n1 \u2212 u\n\n\u2229 k\n2h \u2264 u\n\n(cid:27)(cid:21)\uf8fc\uf8fd\uf8fe\n(cid:27)(cid:21)\uf8fc\uf8fd\uf8fe\n(cid:21)\uf8fc\uf8fd\uf8fe + sup\n\uf8f1\uf8f2\uf8f3 2h(cid:88)\n(cid:20)\n(cid:21)\uf8fc\uf8fd\uf8fe = 2 sup\n2 (cid:99)(cid:88)\n\nk=(cid:100)u2h(cid:101)\n\nu\u2208[0,1]\n\nu\u2208[0,1]\n\n(cid:98) u2h\n\nP\n\nP\n\n, \u03b12 =\n\n(cid:21)\n\nE[f (X, u)]\n\n1\n\n(cid:26)\n(cid:20)\n(cid:20)\n\n1\n\n(cid:18)\n\n1 \u2212 k\nu2h\n\n(cid:19)\n(cid:18) k/2h \u2212 u\n(cid:19)\n(cid:19)\n\n1 \u2212 u\n\n<\n\n1 \u2212 k\nu2h\n\n1 \u2212 k\nu2h\n\nB+\n0\n\nB+\n1\n\n(cid:26)\n(cid:18)\n(cid:18)\n\nB+\n0\n\nB+\n0\n\n(cid:20)\n\nP\n\n<\n\n(cid:19)\n\n\u03b7\u221a\nu\n\n<\n\n\u03b7\u221a\nu\n\n<\n\n\u03b7\u221a\nu\n\n(cid:19)\n\n(cid:18)\n(cid:20)\n\n(cid:18)\n\nB+\n0\n\n1 \u2212 k\nu2h\n\n\u03b7\u221a\nu\n\n<\n\n(cid:19)\n\nP\n\nB+\n0\n\n1 \u2212 k\nu2h\n\n<\n\n\u03b7\u221a\nu\n\n(cid:21)\n\nk=(cid:100)2h\u03b72(cid:101)\n\n, and \u03b14 =\n\n(cid:19)\n\n(cid:18) k/2h \u2212 u\n\n1 \u2212 u\n\nB+\n1\n\n<\n\n\u03b7\u221a\n1 \u2212 u\n\n(cid:21)\uf8fc\uf8fd\uf8fe\n\n{\u03b11 + \u03b12 + \u03b13 + \u03b14},\n\n(cid:20)\n(cid:18)\n(cid:98)u2h(cid:99)(cid:88)\n\nB+\n0\n\n1 \u2212 k\nu2h\n\n(cid:19)\n\n(cid:21)\n\n\u03b7\u221a\nu\n\n1 \u2212 k\nu2h\n\n(cid:19)\n\n(cid:21)\n\n\u00b7\n\n<\n\n\u03b7\u221a\nu\n\n<\n\n(cid:18)\n\n(cid:20)\n\nP\n\nB+\n0\n\nk=(cid:98)u2h(cid:99)\u2212(cid:98)2h\u03b72(cid:99)\n\nE\n\nk=0\n\nk=0\n\n+ sup\nu\u2208[0,1]\n\n\u2264 sup\nu\u2208[0,1]\n\nf : (x0, x1), y \u2192 2h(cid:88)\n\uf8f1\uf8f2\uf8f3 2h(cid:88)\n(cid:20)\n\uf8f1\uf8f2\uf8f3 2h(cid:88)\n\uf8f1\uf8f2\uf8f3(cid:98)u2h(cid:99)(cid:88)\n\uf8f1\uf8f2\uf8f3(cid:98)u2h(cid:99)(cid:88)\n(cid:20)\n(cid:98)2h\u03b72(cid:99)(cid:88)\n(cid:98)u2h(cid:99)\u2212(cid:100)2h\u03b72(cid:101)(cid:88)\n\n= 2 sup\nu\u2208[0,1]\n\n= sup\nu\u2208[0,1]\n\nwith \u03b11 =\n\nk=0\n\nk=0\n\nk=0\n\nk=0\n\nE\n\nP\n\nP\n\n\u03b13 =\n\n2 (cid:101)\nk=(cid:100) u2h\n\nwhere the \ufb01rst two inequalities are obtained by upperbounding the terms exp(\u00b7) by one. Now, we use\nthe above bound to bound \u03b12 + \u03b13,\n\nSince a probability is always upper-bounded by 1, we bound \u03b11 and \u03b14 both by \u03b722h, to get that\n\u03b11 + \u03b14 \u2264 2\u03b722h. We now bound the remaining probabilities appearing in the above expression by\nintegrating over the distribution function of Brownian meander (Durrett et al., 1977, Equation 1.1),\n\nP(cid:2)B+\n0 (t) < x(cid:3) = 2\n\ndw dy\n\n\u221a\n2\n2\u03c0\n\n3\n\n(cid:32)\n\n(cid:33)3\n\n,\n\n(cid:21)\n\nx(cid:112)t(1 \u2212 t)\n(cid:19)\n\n1 \u2212 k\nu2h\n\n<\n\n\u03b7\u221a\nu\n\n(cid:18)\n\nexp(cid:0)\u2212w2/(2(1 \u2212 t))(cid:1)\n(cid:19)\n\n(cid:112)2\u03c0(1 \u2212 t)\n\n\u2264\n\n\u2264\n\n(cid:20)\n\nP\n\nB+\n0\n\n0\n\n0\n\n0\n\nt\n\n2\n\n=\n\ndy\n\n\u2264\n\n2x3\n\n2x3\n\ny2 exp\n\n\u2212 y2\n2t\n\n\u221a\n2\u03c0\n\n(cid:90) y\n\n\u221a\n2\u03c0t\n\ny exp(cid:0)\u2212y2/(2t)(cid:1)\n(cid:90) x\n(cid:18)\n(cid:90) x\nt(cid:112)2\u03c0t(1 \u2212 t)\n3t(cid:112)2\u03c0t(1 \u2212 t)\n(cid:19)\n(cid:18)\n\uf8eb\uf8ed\n(cid:113)\n\uf8eb\uf8ed \u03b7\u221a\nu(cid:113) k\n\uf8eb\uf8ed \u03b7\u221a\nu(cid:113) k\n\n3t(1 \u2212 t)(cid:112)2\u03c0t(1 \u2212 t)\n(cid:21)\n(cid:20)\n(cid:98)u2h(cid:99)\u2212(cid:100)2h\u03b72(cid:101)(cid:88)\n\uf8f6\uf8f83\n(cid:98)u2h(cid:99)\u2212(cid:100)2h\u03b72(cid:101)(cid:88)\n(cid:113) k\n\uf8eb\uf8ed\n(cid:98)u2h(cid:99)\u2212(cid:100)2h\u03b72(cid:101)(cid:88)\nu(cid:113)\n\uf8eb\uf8ed\n2 (cid:101)(cid:88)\n(cid:113) (cid:98)u2h(cid:99)\n\nk=(cid:100) u2h\n2 (cid:101)\n(cid:98)u2h(cid:99)\u2212(cid:100) u2h\n\n\uf8f6\uf8f83\n\uf8f6\uf8f83\n\n\u03b7\u221a\nu\n1 \u2212 k\n\n1 \u2212 k\nu2h\n\nk=(cid:100) u2h\n2 (cid:101)\n\nk=(cid:100) u2h\n2 (cid:101)\n\n\u03b7\u221a\nu\n\n\u221a\n1\n6\n\n\u221a\n1\n\nu2h\n\nu2h\n\nu2h\n\nP\n\n+\n\n<\n\n+\n\n+\n\n+\n\n\u03c0\n\n3\n\n6\n\n\u03c0\n\nB+\n0\n\nk=(cid:100)2h\u03b72(cid:101)\n\nu2h\n\n\u221a\n2\n2\u03c0\n\n3\n\n6\n\n6\n\n\u221a\n1\n\u03c0\n\n\u221a\n1\n\u03c0\n\n\u221a\n2\n2\u03c0\n\n\u03b7\u221a\n1 \u2212 k\n\nu2h\n\n\u03b7\u221a\nu\n\n\uf8eb\uf8ed\n(cid:113)\n\uf8f6\uf8f83\n\nu2h + k\nu2h\n\n\uf8f6\uf8f83\n\n\u00b7\n\n\uf8f6\uf8f83\n\n\u03b7\u221a\nu\n1 \u2212 k\n\nu2h\n\n(cid:113) k\n\nu2h\n\n7\n\n(cid:98) u2h\n\nk=(cid:100)2h\u03b72(cid:101)\n\n(cid:98) u2h\n\nk=(cid:100)2h\u03b72(cid:101)\n\n(cid:98) u2h\n\n2 (cid:99)(cid:88)\n2 (cid:99)(cid:88)\n2 (cid:99)(cid:88)\n2 (cid:99)(cid:88)\n\nk=(cid:100)2h\u03b72(cid:101)\n\n(cid:98) u2h\n\nk=(cid:100)2h\u03b72(cid:101)\n\n\u03b12 + \u03b13 =\n\n\u2264\n\n\u2264\n\n\u2264\n\n\fChanging the indexation as k = \u2212k(cid:48) + (cid:98)u2h(cid:99), we get\n\n\u03b12 + \u03b13 \u2264\n\n1\nk3/2 +\n\n\u2264\n\nk3/2 \u2264\nwhere in the last line we used that for any k0 \u2265 1,\n\n\u221a\n\u03c0\n\n1\n\n3\n\n\u221e(cid:88)\n\nk=k0\n\n(cid:0)2h\u03b72(cid:1)3/2\n(cid:0)2h\u03b72(cid:1)3/2\n\n\u221a\n\u03c0\n\n6\n\n\u221e(cid:88)\n\nk=(cid:100)2h\u03b72(cid:101)\n\n\uf8eb\uf8ec\uf8ed (cid:98) u2h\n2 (cid:99)(cid:88)\n\u221e(cid:88)\n(cid:90) k\n\nk=(cid:100)2h\u03b72(cid:101)\n\n1\n\nk3/2\n\nk=(cid:100)2h\u03b72(cid:101)\n\n(cid:98)u2h(cid:99)\u2212(cid:100) u2h\n\n\uf8f6\uf8f7\uf8f8\n2 (cid:101)(cid:88)\n(cid:0)2h\u03b72(cid:1)3/2\n3(cid:112)(cid:100)2h\u03b72(cid:101) \u2264 1\u221a\n(cid:90) \u221e\n\n\u221a\n\u03c0\n\n3\n\n\u03b722h \u2264 \u03b722h,\n\n\u03c0\n\n1\n\nk3/2 \u2264 1\nk3/2\n0\n\n+\n\nk=k0+1\n\nk\u22121\n\n1\nu3/2 du =\n\n1\nk3/2\n0\n\n+\n\nk0\n\n1\nu3/2 du =\n\n1\nk3/2\n0\n\n+\n\n2\u221a\nk0\n\n\u2264 3\u221a\nk0\n\n\u00b7\n\nWe \ufb01nally have\n\nand therefore\n\n\u2200u, \u03b11 + \u03b12 + \u03b13 + \u03b14 \u2264 3\u03b722h\n\nE[Nh(\u03b7)] \u2264 6\u03b722h.\n\nTo conclude the analysis, we put Lemma 2 and Lemma 3 together in order to bound the sample\ncomplexity conditioned on event C.\nLemma 4. There exists c > 0 such that for all \u03b5 \u2264 1/2, E[N\u03b5 | C ] \u2264 c log2(1/\u03b5).\n\nProof. By de\ufb01nition of hmax,\n\n\u03b52 \u2264\n\n5\n\n2 \u00d7 2hmax\n\nln\n\n(cid:18) 2hmax\n\n(cid:19)\n\n\u03b5\n\n\u2264 5\n2\n\n(cid:112)2hmax /\u03b5\n\n2hmax\n\n,\n\nfrom which we deduce that\n\u221a\n\u03b5\u03b52 \u2264\n\n5\n\n2 \u00d7 2hmax/2\n\nand therefore 2hmax \u2264 25\n4\u03b55\n\n,\n\nwhich gives us an upper bound on hmax,\n\nFurthermore, using Lemma 1, we get that for any \u03b5 \u2264 1/2,\n\nhmax \u2264 ln(25/4)\n\n+\n\n5 ln(1/\u03b5)\n\n= O(log(1/\u03b5)).\n\nln 2\n\nln 2\n\nE[Nh(\u03b7h)] = E[Nh(\u03b7h)| C ]P[C] + E[Nh(\u03b7h)| Cc]P[Cc] \u2265 E[Nh(\u03b7h)| C ](1 \u2212 \u03b5) \u2265 E[Nh(\u03b7h)|C ]\n\n2\n\n\u00b7\n\nE[N\u03b5| C ] \u2264 2\n\nWe now use Lemma 2 and Lemma 3 to get\n\n(cid:34)\nhmax(cid:88)\nhmax+1(cid:88)\n\nh=0\n\nE\n\nh=1\n\nNh\n\n2h\n\n5\n2\n\n(cid:17)\n\n(cid:16)\n\n\u2264 60\n\nln(2h+1/\u03b5)\n\nhmax(cid:88)\n\n(cid:33)(cid:12)(cid:12)(cid:12)(cid:12) C\n(cid:32)(cid:114)\n(cid:35)\n(ln(1/\u03b5) + h ln 2) = O(cid:0)h2\nmax + hmax log(1/\u03b5)(cid:1) = O(cid:0)log2(1/\u03b5)(cid:1).\n(cid:0)1/2h(cid:1) \u2264 \u03b5. Then, we combine it with Lemma 1,\n\n2h+1/\u03b5\n\nh=0\n\nln\n\nWe also bound the sample complexity on Cc, the event complementary to C, by the total number\nof possible intervals [k/2h, (k + 1)/2h] with \u03b7\u03b5\nLemma 2, and Lemma 3 to get Proposition 2 which is the second part of the main theorem.\nProposition 2. There exists c > 0 such that for all \u03b5 < 1/2,\nE[N\u03b5] \u2264 c log2(1/\u03b5).\n\n= 60\n\nProof. From the law of total expectation,\n\nE[N\u03b5] = E[N\u03b5| C ]P[C] + E[N\u03b5|Cc]P[Cc] \u2264 E[N\u03b5| C ] + \u03b55E[N\u03b5|Cc].\n\n8\n\n\fpoints are evaluated. Then,\n\nBy Lemma 4, we have that E[N\u03b5| C ] = O(cid:0)log2(1/\u03b5)(cid:1). Now let hmax be the maximum h at which\nAs in the proof of Lemma 4, we get 2hmax+1 = O(cid:0)\u03b5\u22125(cid:1). We \ufb01nally obtain that\n\nN\u03b5 \u2264 hmax(cid:88)\nE[N\u03b5] \u2264 O(cid:0)log2(1/\u03b5)(cid:1) + O(1) = O(cid:0)log2(1/\u03b5)(cid:1).\n\n2h = 2hmax+1 \u2212 1 \u2264 2hmax+1.\n\nh=0\n\nProposition 2 together with Proposition 1 establish the proof of Theorem 1, the holy grail.\n\n5 Numerical evaluation of OOB\n\nFor an illustration, we ran a simple experiment and for different values of \u03b5, we computed the average\nempirical sample complexity N\u03b5 on 250 independent runs that you can see on the left plot. We also\nplot one point for each run of OOB instead of averaging the sample complexity, to be seen on the right.\nThe experiment indicates a linear dependence between the sample complexity and log2(1/\u03b5).\n\n6 Conclusion and ideas for extensions and generalizations\n\nWe presented OOB, an algorithm inspired by DOO (Munos, 2011) that ef\ufb01ciently optimizes a Brownian\n\nmotion. We proved that the sample complexity of OOB is of order O(cid:0)log2(1/\u03b5)(cid:1) which improves\nthe Brownian motion optimization, we do not know whether O(cid:0)log2(1/\u03b5)(cid:1) is the minimax rate of\n\nover the previously best known bound (Calvin et al., 2017). As we are not aware of a lower bound for\n\nthe sample complexity or if there exists an algorithm that can do even better.\nWhat would be needed to do if we wanted to use our approach for Gaussian processes with a different\nkernel? The optimistic approach we took is quite general and only Lemma 3 would need additional\nwork as this is the ingredient most speci\ufb01c to Brownian motion. Notice, that Lemma 3 bounds the\nnumber of near-optimal nodes of a Brownian motion in expectation. To bound the expected number\nof near-optimal nodes, we use the result of \u0414\u0435\u043d\u0438\u0441\u043e\u0432 (1983) which is based on 2 components:\n\n1 A Brownian motion can be rewritten as an af\ufb01ne transformation of a Brownian motion\nconditioned to be positive, translated by an (independent) time at which the Brownian\nmotion attains its maximum.\n\n2 A Brownian motion conditioned to be positive is a Brownian meander. It requires some\nadditional work to prove that a Brownian motion conditioned to be positive is actually\nproperly de\ufb01ned.\n\nA similar result for another Gaussian process or its generalization of our result to a larger class of\nGaussian processes would need to adapt or generalize these two items in Lemma 3. On the other hand,\nthe adaptation or generalization of the other parts of the proof would be straightforward. Moreover,\nfor the second item, the full law of the process conditioned to be positive is actually not needed, only\nthe local time of the Gaussian process conditioned to be positive at points near zero.\n\n9\n\n\fAcknowledgements This research was supported by European CHIST-ERA project DELTA,\nFrench Ministry of Higher Education and Research, Nord-Pas-de-Calais Regional Council, Inria and\nOtto-von-Guericke-Universit\u00a8at Magdeburg associated-team north-European project Allocate, French\nNational Research Agency projects ExTra-Learn (n.ANR-14-CE24-0010-01) and BoB (n.ANR-16-\nCE23-0003), FMJH Program PGMO with the support to this program from Criteo, a doctoral grant\nof \u00b4Ecole Normale Sup\u00b4erieure, and Maryse & Michel Grill.\n\nReferences\nHisham Al-Mharmah and James M. Calvin. Optimal random non-adaptive algorithm for global\n\noptimization of Brownian motion. Journal of Global Optimization, 8(1):81\u201390, 1996.\n\nKinjal Basu and Souvik Ghosh. Analysis of Thompson sampling for Gaussian process optimization\n\nin the bandit setting. arXiv preprint arXiv:1705.06808, 2018.\n\nJames M. Calvin, Mario Hefter, and Andr\u00b4e Herzwurm. Adaptive approximation of the minimum of\n\nBrownian motion. Journal of Complexity, 39(C):17\u201337, 2017.\n\nOlivier Chapelle and Lihong Li. An empirical evaluation of Thompson sampling.\n\nInformation Processing Systems. 2011.\n\nIn Neural\n\n\u0418. \u0412. \u0414\u0435\u043d\u0438\u0441\u043e\u0432. \u0421\u043b\u0443\u0447\u0430\u0439\u043d\u043e\u0435 \u0431\u043b\u0443\u0436\u0434\u0430\u043d\u0438\u0435 \u0438 \u0432\u0438\u043d\u0435\u0440\u043e\u0432\u0441\u043a\u0438\u0439 \u043f\u0440\u043e\u0446\u0435\u0441\u0441, \u0440\u0430\u0441\u0441\u043c\u0430\u0442\u0440\u0438\u0432\u0430\u0435\u043c\u044b\u0435 \u0438\u0437 \u0442\u043e\u0447\u043a\u0438\n\n\u043c\u0430\u043a\u0441\u0438\u043c\u0443\u043c\u0430. \u0422\u0435\u043e\u0440\u0438\u044f \u0432\u0435\u0440\u043e\u044f\u0442\u043d\u043e\u0441\u0442\u0435\u0439 \u0438 \u0435\u0435 \u043f\u0440\u0438\u043c\u0435\u043d\u0435\u043d\u0438\u044f, 28(4):785\u2013788, 1983.\n\nRichard T. Durrett, Donald L. Iglehart, and Douglas R. Miller. Weak convergence to Brownian\n\nmeander and Brownian excursion. The Annals of Probability, 5(1):117\u2013129, 1977.\n\nMario Hefter and Andr\u00b4e Herzwurm. Optimal strong approximation of the one-dimensional squared\n\nBessel process. Communications in Mathematical Sciences, 15(8):2121\u20132141, 2017.\n\nR\u00b4emi Munos. Optimistic optimization of deterministic functions without the knowledge of its\n\nsmoothness. In Neural Information Processing Systems, 2011.\n\nDaniel Russo, Benjamin Van Roy, Abbas Kazerouni, Ian Osband, and Zheng Wen. A tutorial on\n\nThompson sampling. Foundations and Trends in Machine Learning, 2018.\n\nWilliam R. Thompson. On the likelihood that one unknown probability exceeds another in view of\n\nthe evidence of two samples. Biometrika, 25:285\u2013294, 1933.\n\n10\n\n\f", "award": [], "sourceid": 1561, "authors": [{"given_name": "Jean-Bastien", "family_name": "Grill", "institution": "Google DeepMind"}, {"given_name": "Michal", "family_name": "Valko", "institution": "Inria Lille - Nord Europe"}, {"given_name": "Remi", "family_name": "Munos", "institution": "DeepMind"}]}