This paper presents a latent variational framework for designing stochastic optimization algorithms using ideas from stochastic control. The main contribution of the paper is an action functional, such that the corresponding Euler-Lagrange (EL) equations give rise to a system of Forward-Backward stochastic differential equations (FB-SDEs). These equations are generalizations of the ODEs for deterministic optimization obtained by Wibisono et al., 2016. The paper also presents an analysis of the rate of convergence. The reviewers are uniformly positive about this work, and the authors' response has addressed most of their concerns.