Modern life is full of challenging optimization problems that we unknowingly attempt to solve. For instance, a common dilemma often encountered is the decision of picking a parking spot while trying to minimize both the distance to the goal destination and time spent searching for parking; one strategy is to drive as close as possible to the goal destination but risk a penalty cost if no parking spaces can be found. Optimization problems of this class all have underlying time-varying processes that can be altered by a decision/input to minimize some cost. Such optimization problems are commonly solved by a class of methods called Dynamic Programming (DP) that breaks down a complex optimization problem into a simpler family of sub-problems. In the 1950s Richard Bellman introduced a class of DP methods that broke down Multi-Stage Optimization Problems (MSOP) into a nested sequence of ``tail problems”. Bellman showed that for any MSOP with a cost function that satisfies a condition called additive separability, the solution to the tail problem of the MSOP initialized at time-stage k>0 can be used to solve the tail problem initialized at time-stage k-1. Therefore, by recursively solving each tail problem of the MSOP, a solution to the original MSOP can be found. This dissertation extends Bellman`s theory to a broader class of MSOPs involving non-additively separable costs by introducing a new state augmentation solution method and generalizing the Bellman Equation. This dissertation also considers the analogous continuous-time counterpart to discrete-time MSOPs, called Optimal Control Problems (OCPs). OCPs can be solved by solving a nonlinear Partial Differential Equation (PDE) called the Hamilton-Jacobi-Bellman (HJB) PDE. Unfortunately, it is rarely possible to obtain an analytical solution to the HJB PDE. This dissertation proposes a method for approximately solving the HJB PDE based on Sum-Of-Squares (SOS) programming. This SOS algorithm can be used to synthesize controllers, hence solving the OCP, and also compute outer bounds of reachable sets of dynamical systems. This methodology is then extended to infinite time horizons, by proposing SOS algorithms that yield Lyapunov functions that can approximate regions of attraction and attractor sets of nonlinear dynamical systems arbitrarily well.

In this dissertation, new data-driven techniques are developed to solve three problems related to generating predictive models of the immune system. These problems and their solutions are summarized as follows. The first problem is that, while cellular characteristics can be measured using flow cytometry, immune system cells are often analyzed only after they are sorted into groups by those characteristics. In Chapter 3 a method of analyzing the cellular characteristics of the immune system cells by generating Probability Density Functions (PDFs) to model the flow cytometry data is proposed. To generate a PDF to model the distribution of immune cell characteristics a new class of random variable called Sliced-Distributions (SDs) is developed. It is shown that the SDs can outperform other state-of-the-art methods on a set of benchmarks and can be used to differentiate between immune cells taken from healthy patients and those with Rheumatoid Arthritis. The second problem is that while immune system cells can be broken into different subpopulations, it is unclear which subpopulations are most significant. In Chapter 4 a new machine learning algorithm is formulated and used to identify subpopulations that can best predict disease severity or the populations of other immune cells. The proposed machine learning algorithm performs well when compared to other state-of-the-art methods and is applied to an immunological dataset to identify disease-relevant subpopulations of immune cells denoted immune states. Finally, while immunotherapies have been effectively used to treat cancer, selecting an optimal drug dose and period of treatment administration is still an open problem. In Chapter 5 a method to estimate Lyapunov functions of a system with unknown dynamics is proposed. This method is applied to generate a semialgebraic set containing immunotherapy doses and period of treatment that is predicted to eliminate a patient's tumor. The problem of selecting an optimal pulsed immunotherapy treatment from this semialgebraic set is formulated as a Global Polynomial Optimization (GPO) problem. In Chapter 6 a new method to solve GPO problems is proposed and optimal pulsed immunotherapy treatments are identified for this system.

The lack of healthy behaviors - such as physical activity and balanced diet - in

modern society is responsible for a large number of diseases and high mortality rates in

the world. Adaptive behavioral interventions have been suggested as a way to promote

sustained behavioral changes to address these issues. These adaptive interventions

can be modeled as closed-loop control systems, and thus applying control systems

engineering and system identification principles to behavioral settings might provide

a novel way of improving the quality of such interventions.

Good understanding of the dynamic processes involved in behavioral experiments

is a fundamental step in order to design such interventions with control systems ideas.

In the present work, two different behavioral experiments were analyzed under the

light of system identification principles and modelled as dynamic systems.

In the first study, data gathered over the course of four days served as the basis for

ARX modeling of the relationship between psychological constructs (negative affect

and self-efficacy) and the intensity of physical activity. The identified models suggest

that this behavioral process happens with self-regulation, and that the relationship

between negative affect and self-efficacy is represented by a second order underdamped

system with negative gain, while the relationship between self-efficacy and physical

activity level is an overdamped second order system with positive gain.

In the second study, which consisted of single-bouts of intense physical activity,

the relation between a more complex set of behavioral variables was identified as a

semi-physical model, with a theoretical set of system equations derived from behavioral

theory. With a prescribed set of physical activity intensities, it was found that less fit

participants were able to get higher increases in affective state, and that self-regulation

processes are also involved in the system.

Behavioral health problems such as physical inactivity are among the main causes of mortality around the world. Mobile and wireless health (mHealth) interventions offer the opportunity for applying control engineering concepts in behavioral change settings. Social Cognitive Theory (SCT) is among the most influential theories of health behavior and has been used as the conceptual basis of many behavioral interventions. This dissertation examines adaptive behavioral interventions for physical inactivity problems based on SCT using system identification and control engineering principles. First, a dynamical model of SCT using fluid analogies is developed. The model is used throughout the dissertation to evaluate system identification approaches and to develop control strategies based on Hybrid Model Predictive Control (HMPC). An initial system identification informative experiment is designed to obtain basic insights about the system. Based on the informative experimental results, a second optimized experiment is developed as the solution of a formal constrained optimization problem. The concept of Identification Test Monitoring (ITM) is developed for determining experimental duration and adjustments to the input signals in real time. ITM relies on deterministic signals, such as multisines, and uncertainty regions resulting from frequency domain transfer function estimation that is performed during experimental execution. ITM is motivated by practical considerations in behavioral interventions; however, a generalized approach is presented for broad-based multivariable application settings such as process control. Stopping criteria for the experimental test utilizing ITM are developed using both open-loop and robust control considerations.

A closed-loop intensively adaptive intervention for physical activity is proposed relying on a controller formulation based on HMPC. The discrete and logical features of HMPC naturally address the categorical nature of the intervention components that include behavioral goals and reward points. The intervention incorporates online controller reconfiguration to manage the transition between the behavioral initiation and maintenance training stages. Simulation results are presented to illustrate the performance of the system using a model for a hypothetical participant under realistic conditions that include uncertainty. The contributions of this dissertation can ultimately impact novel applications of cyberphysical system in medical applications.