We present a theoretical framework, based on differential mean field games, for expressing diel vertical migration in the ocean as a game with a continuum of players. In such a game, each agent partially controls its own state by adjusting its vertical velocity but the vertical position in a water column is also subject to random fluctuations. A representative player has to make decisions based on aggregated information about the states of the other players. For this vertical differential game, we derive a mean field system of partial differential equations for finding a Nash equilibrium for the whole population. It turns out that finding Nash equilibria in the game is equivalent to solving a PDE-constrained optimization problem. We detail this equivalence when the expected fitness of the representative player can be approximated with a constant and solve both formulations numerically. We illustrate the results on simple numerical examples and construct several test cases to compare the two analytical approaches.

The rigorous connection between viscosity solutions for the Bellman equation in optimal control and the Pontryagin Maximum Principle has been established. The method developed for controlled ordinary differential equations was extended to infinite dimensions to derive the Pontryagin principle for 1) a class of controlled nonlinear evolution equations in a Hilbert space, 2) a class of controlled nonlinear, divergence form parabolic partial differential equations; and 3) a class of differential-difference equations. Additional subjects were studied were the extension of the idea of viscosity solution to equations with only time-measureable Hamiltonians and the optimal cooling of a free boundary problem with Stefan problem dynamics. Two problems of interest in specific applications were solved. The optimal control is characterized in the class of monotone functions which minimizes the H1 distance to a given function. This problem, a specific monotone follower problem, arises in production planning. An optimal portfolio selection problem is considered which includes stock, options, bonds and borrowed cash at an interest rate different from the bond interest rate. This problem is formulated using stochastic optimal control and explicitly constructed the solution of the Bellman equation. The objective of the study was to derive the option price which the market sets to minimize the investor's maximal expected utility of wealth.

Abstract We propose a method to carry out an inequality assessment in a dynamic and cross-sectional framework, by applying the dynamic version of a suitable inequality index, such as the Gini coefficient, as a function of time. We use our methodology to a setup where the optimal value functions is the individuals’ income flows while the initial conditions characterize their level of wealth. When the Hamilton–Jacobi–Bellman system of equations can be solved in closed form, the monotone path of the income distribution is established. Extending the model according to a government intervention gives the possibility to study, first policy for reducing income inequality under a specific exogenous target, and second to minimise income inequality across individuals.

We consider a multi-agent system consisting of several populations. The interaction between large populations of agents seeking to regulate their state on the basis of the distribution of the neighboring populations is studied. Examples of such interactions can typically be found in social networks and opinion dynamics, where heterogeneous agents or clusters are present and decisions are influenced by individual objectives as well as by global factors. In this paper, such a problem is posed as a multi-population mean-field game, for which solutions depend on two partial differential equations, namely the Hamilton-Jacobi-Bellman equation and the Fokker-Planck-Kolmogorov equation. The case in which the distributions of agents are sums of polynomials and the value functions are quadratic polynomials is considered. It is shown that for this class of problems, which can be considered as approximations of more general problems, a set of ordinary differential equations, with two-point boundary value conditions, can be solved in place of the more complicated partial differential equations characterizing the solution of the multi-population mean-field game.

The optimal correction of a material point’s trajectory under small perturbations is an important problem in control theory. The study of such processes can be reduced to solving a boundary value problem for a special nonlinear second-order partial differential equation known as the Bellman equation. This paper deals with a partial differential equation with quadratic nonlinearity, which is a special case of the Bellman equation. The equation contains three independent variables—one temporal and two spatial, and two arbitrary time-dependent functions as multipliers. Multidimensionality is a distinctive feature of this equation, which significantly complicates its analytical study. Therefore, we use Lie group analysis which is an effective technique to analytically study nonlinear partial differential equations. It allows the investigation of not only the symmetry properties of equations, but also to find their particular solutions. The problem of group classification for this Bellman equation is solved with respect to two arbitrary time-dependent functions. It is established that in the case of the arbitrariness of these functions, the equation admits a four-parameter group of point transformations. This group expands to five-parameter and six-parameter groups for the parametric representation and linear dependency of the functions, respectively. Several invariant solutions are also constructed.

used to solve some infinite-horizon, autonomous Bellman equations. The Bellman equation can be solved by backwards induction, either analytically in a few A Bellman equation, named after Richard E. Bellman, is a technique in dynamic programming which breaks an optimization problem into a sequence of simpler subproblems, as Bellman's "principle of optimality" prescribes. It is a necessary condition for optimality. The "value" of a decision problem at a certain point in time is written in terms of the payoff from some initial choices and the "value" A Bellman equation, named after Richard E. Bellman, is a technique in dynamic programming which breaks an optimization problem into a sequence of simpler subproblems, as Bellman's "principle of optimality" prescribes. It is a necessary condition for optimality. The "value" of a decision problem at a certain point in time is written in terms of the payoff from some initial choices and the "value" of the remaining decision problem that results from those initial choices. The equation applies to algebraic structures with a total ordering; for algebraic structures with a partial ordering, the generic Bellman's equation can be used. The Bellman equation was first applied to engineering control theory and to other topics in applied mathematics, and subsequently became an important tool in economic theory; though the basic concepts of dynamic programming are prefigured in John von Neumann and Oskar Morgenstern's Theory of Games and Economic Behavior and Abraham Wald's sequential analysis. The term "Bellman equation" usually refers to the dynamic programming equation (DPE) associated with discrete-time optimization problems. In continuous-time optimization problems, the analogous equation is a partial differential equation called the Hamilton–Jacobi–Bellman equation. In discrete time any multi-stage optimization problem can be solved by analyzing the appropriate Bellman equation. The appropriate Bellman equation can be found by introducing new state variables (state augmentation). However, the resulting augmented-state multi-stage… The Bellman equation is classified as a functional equation, because solving it means finding the unknown function V {\displaystyle V} , which is the value function. Recall that the value function describes the best possible value of the objective, as a function of the state x {\displaystyle x} . By calculating the value function, we will also find the function a ( x ) {\displaystyle a(x)} that describes the optimal action as a function of the state; this is called the policy function. Under…

Исследуется уравнение с квадратичной нелинейностью, которое является частным случаем дифференциального уравнения типа Беллмана. В результате решения задачи групповой классификации установлено, что в случае произвольного вида двух функций, входящих в нелинейные члены, уравнение допускает четырехпараметрическую группу преобразований, которая расширяется до пятипараметрической и шестипараметрической в случае параметрической и линейной зависимости функций. Построены некоторые инвариантные решения. The optimal correction of a material point’s trajectory under small perturbations is an important problem in control theory. The study of such processes can be reduced to solving a boundary value problem for a special nonlinear second-order partial differential equation known as the Bellman equation. This paper deals with a partial differential equation with quadratic nonlinearity, which is a special case of the Bellman equation. The equation contains three independent variables—one temporal and two spatial, and two arbitrary time-dependent functions as multipliers. Multidimensionality is a distinctive feature of this equation, which significantly complicates its analytical study. Therefore, we use Lie group analysis which is an effective technique to analytically study nonlinear partial differential equations. It allows the investigation of not only the symmetry properties of equations, but also to find their particular solutions. The problem of group classification for this Bellman eq

Optimal Quantum Feedback Control for Canonical Observables We show that the stochastic Schrodinger equation for the filtered state of a system, with linear free dynamics, undergoing continual non-demolition measurement or either position or momentum, or both together, can be solved explicitly within a class of Gaussian states which we call extended coherent states. The asymptotic limit yields a class of relaxed states which we describe explicitly. Bellman's principle is then applied directly to optimal feedback control of such dynamical systems and the Hamilton Jacobi Bellman equation for the minimum cost is derived. The situation of quadratic performance criteria is treated as the important special case and solved exactly for the class of relaxed states. Published as: Quantum Stochastics and Information: Statistics, Filtering & Control, pp. 262-279 Eds. M. Guta and V.P. Belavkin, World Scientific 2008 arXiv categories: quant-ph