A framework for online decision making under uncertainty and time constraints, with online stochastic algorithms for implementing the framework, performance guarantees, and demonstrations of a variety of applications.