Abstract
A discrete-time method for solving problems in optimal quantum control is presented. Controlling the time discretized markovian dynamics of a quantum system can be reduced to a Markov-decision process. We demonstrate this method in this with a class of simple one qubit systems, which are also discretized in space. For the task of state preparation we solve the examples both numerically and analytically with dynamic programming techniques.