We provide several quantum algorithms for continuous optimization that do not require gradient estimation. Instead, we encode the optimization problem into the dynamics of a physical system and coherently simulate the time evolution. We focus on the setting where the objective function can only be accessed via a phase oracle. Our first two algorithms can find local optima of a differentiable function (f: \mathbb{R}^N \rightarrow \mathbb{R}) by simulating either classical or quantum dynamics with friction via a time-dependent Hamiltonian. We show that for the benchmark problem of optimizing a locally quadratic objective function, these methods require a total of (O(N^2\kappa^2/h_x^2\epsilon)) queries to a phase oracle to find an (\epsilon)-approximate local optimum, where (\kappa) is the condition number of the Hessian matrix and (h_x) is the discretization spacing. In contrast, we show that methods based on gradient descent require (O(N^{3/2}(1/\epsilon)^{\kappa log(3)/4})) queries. This corresponds to an exponential separation between the query upper bounds for the benchmark problem. Our third algorithm can find the global optimum of (f) by preparing a classical low-temperature thermal state via simulation of the classical Liouvillian operator associated with the Nos'e Hamiltonian. We use results from the quantum thermodynamics literature to bound the thermalization time for the discrete system. Additionally, we analyze barren plateau effects that commonly plague quantum optimization algorithms and observe that our approach is vastly less sensitive to this problem than standard gradient-based optimization. Our results suggests that these dynamical optimization approaches may be far more scalable for future quantum machine learning, optimization and variational experiments than was widely believed.