From simple growth to numerical simulations : a primer in dynamic programming

These notes provide an intuitive introduction to dynamic programming. The first two Sections, which can be skipped, present the standard deterministic Ramsey model using the Lagrangian approach. Section 3 reformulates the Ramsey problem by means of a Bellman equation, while Section 4 shows how to "guess" the maximum value function solving the problem (when this is possible). Section 5 is devoted to applications of the envelope theorem. Section 6 provides a "paper and pencil" introduction to the numerical techniques used in dynamic programming, and can be skipped by the uninterested reader. Sections 7 to 9 are devoted to stochastic modelling, and to the use of stochastic Bellman equations. Section 10 extends the discussion of numerical techniques. Two Appendixes provide details about the Matlab routines used to deal with the examples, and the solutions of the exercises proposed in the main text.