Physical activity helps in reducing the risk of many chronic diseases, and plays a key role in maintaining good health of an individual. Just Walk is an intensively adaptive physical activity intervention, which has been designed based on system identification and control engineering principles. The goal of Just Walk is to design interventions that are responsive to an individual's changing needs, and thus encourage the individual to increase the number of steps walked.

Regularization is widely used in the field of machine learning. The goal of this thesis is to see how classical system identification principles in combination with machine learning methods like regularization help towards getting improved model estimates for complex systems. Estimating individual behavioral models using traditional prediction error methods can be done using an order selection. However, this method is can be computationally expensive due to the extensive search performed on a large set of order combination. If order selection is not done properly, it can cause bias (low order) and variance (high order) issues. In such cases regularization plays an important role in addressing the bias-variance trade-off.

One of the most important applications of identifying individual behavioral models is to understand what factors impact most the behavior of the person. Here "factors" can be considered as inputs (designed or environmental) to the participant over the course of the study, and the "behavior" is the step count of the participant under study. This is done by estimating models with different input combinations and then seeing which combinations of inputs (influence behavior most) give the best model estimate (best describe behavior of the person). As a part of this thesis, it is studied how regularized models can give a better estimation of personalized behavioral models, for the Just Walk study, which can further help in designing personalized interventions.

This work is concerned with how best to reconstruct images from limited angle tomographic measurements. An introduction to tomography and to limited angle tomography will be provided and a brief overview of the many fields to which this work may contribute is given.

The traditional tomographic image reconstruction approach involves Fourier domain representations. The classic Filtered Back Projection algorithm will be discussed and used for comparison throughout the work. Bayesian statistics and information entropy considerations will be described. The Maximum Entropy reconstruction method will be derived and its performance in limited angular measurement scenarios will be examined.

Many new approaches become available once the reconstruction problem is placed within an algebraic form of Ax=b in which the measurement geometry and instrument response are defined as the matrix A, the measured object as the column vector x, and the resulting measurements by b. It is straightforward to invert A. However, for the limited angle measurement scenarios of interest in this work, the inversion is highly underconstrained and has an infinite number of possible solutions x consistent with the measurements b in a high dimensional space.

The algebraic formulation leads to the need for high performing regularization approaches which add constraints based on prior information of what is being measured. These are constraints beyond the measurement matrix A added with the goal of selecting the best image from this vast uncertainty space. It is well established within this work that developing satisfactory regularization techniques is all but impossible except for the simplest pathological cases. There is a need to capture the "character" of the objects being measured.

The novel result of this effort will be in developing a reconstruction approach that will match whatever reconstruction approach has proven best for the types of objects being measured given full angular coverage. However, when confronted with limited angle tomographic situations or early in a series of measurements, the approach will rely on a prior understanding of the "character" of the objects measured. This understanding will be learned by a parallel Deep Neural Network from examples.

Robotic lower limb prostheses provide new opportunities to help transfemoral amputees regain mobility. However, their application is impeded by that the impedance control parameters need to be tuned and optimized manually by prosthetists for each individual user in different task environments. Reinforcement learning (RL) is capable of automatically learning from interacting with the environment. It becomes a natural candidate to replace human prosthetists to customize the control parameters. However, neither traditional RL approaches nor the popular deep RL approaches are readily suitable for learning with limited number of samples and samples with large variations. This dissertation aims to explore new RL based adaptive solutions that are data-efficient for controlling robotic prostheses.

This dissertation begins by proposing a new flexible policy iteration (FPI) framework. To improve sample efficiency, FPI can utilize either on-policy or off-policy learning strategy, can learn from either online or offline data, and can even adopt exiting knowledge of an external critic. Approximate convergence to Bellman optimal solutions are guaranteed under mild conditions. Simulation studies validated that FPI was data efficient compared to several established RL methods. Furthermore, a simplified version of FPI was implemented to learn from offline data, and then the learned policy was successfully tested for tuning the control parameters online on a human subject.

Next, the dissertation discusses RL control with information transfer (RL-IT), or knowledge-guided RL (KG-RL), which is motivated to benefit from transferring knowledge acquired from one subject to another. To explore its feasibility, knowledge was extracted from data measurements of able-bodied (AB) subjects, and transferred to guide Q-learning control for an amputee in OpenSim simulations. This result again demonstrated that data and time efficiency were improved using previous knowledge.

While the present study is new and promising, there are still many open questions to be addressed in future research. To account for human adaption, the learning control objective function may be designed to incorporate human-prosthesis performance feedback such as symmetry, user comfort level and satisfaction, and user energy consumption. To make the RL based control parameter tuning practical in real life, it should be further developed and tested in different use environments, such as from level ground walking to stair ascending or descending, and from walking to running.

Stroke remains a leading cause of adult disability in the United States. In recent studies, chronic vagus nerve stimulation (VNS) has been proven to enhance functional recovery when paired with motor rehabilitation training after stroke. Other studies have also demonstrated that delivering VNS during the onset of a stroke may elicit some neuroprotective effects as observed in remaining neural tissue and motor function. While these studies have demonstrated the benefits of VNS as a treatment or therapy in combatting stroke damage, the mechanisms responsible for these effects are still not well understood or known. The aim of this research was to further investigate the mechanisms underlying the efficacy of acute VNS treatment of stroke by observing the effect of VNS when applied after the onset of stroke. Animals were randomly assigned to three groups: Stroke animals received cortical ischemia (ET-1 injection), VNS+Stroke animals received acute VNS starting within 48 hours after cortical ischemia and continuing once per day for three days, or Control animals which received neither the injury nor stimulation. Results showed that stroke animals receiving acute VNS had smaller lesion volumes and larger motor cortical maps than those in the Stroke group. The results suggest VNS may confer neuroprotective effects when delivered within the first 96 hours of stroke.

Spike sorting is a critical step for single-unit-based analysis of neural activities extracellularly and simultaneously recorded using multi-channel electrodes. When dealing with recordings from very large numbers of neurons, existing methods, which are mostly semiautomatic in nature, become inadequate.

This dissertation aims at automating the spike sorting process. A high performance, automatic and computationally efficient spike detection and clustering system, namely, the M-Sorter2 is presented. The M-Sorter2 employs the modified multiscale correlation of wavelet coefficients (MCWC) for neural spike detection. At the center of the proposed M-Sorter2 are two automatic spike clustering methods. They share a common hierarchical agglomerative modeling (HAM) model search procedure to strategically form a sequence of mixture models, and a new model selection criterion called difference of model evidence (DoME) to automatically determine the number of clusters. The M-Sorter2 employs two methods differing by how they perform clustering to infer model parameters: one uses robust variational Bayes (RVB) and the other uses robust Expectation-Maximization (REM) for Student’s 𝑡-mixture modeling. The M-Sorter2 is thus a significantly improved approach to sorting as an automatic procedure.

M-Sorter2 was evaluated and benchmarked with popular algorithms using simulated, artificial and real data with truth that are openly available to researchers. Simulated datasets with known statistical distributions were first used to illustrate how the clustering algorithms, namely REMHAM and RVBHAM, provide robust clustering results under commonly experienced performance degrading conditions, such as random initialization of parameters, high dimensionality of data, low signal-to-noise ratio (SNR), ambiguous clusters, and asymmetry in cluster sizes. For the artificial dataset from single-channel recordings, the proposed sorter outperformed Wave_Clus, Plexon’s Offline Sorter and Klusta in most of the comparison cases. For the real dataset from multi-channel electrodes, tetrodes and polytrodes, the proposed sorter outperformed all comparison algorithms in terms of false positive and false negative rates. The software package presented in this dissertation is available for open access.

Unmanned aerial vehicles (UAVs) are widely used in many applications because of their small size, great mobility and hover performance. This has been a consequence of the fast development of electronics, cheap lightweight flight controllers for accurate positioning and cameras. This thesis describes modeling, control and design of an oblique-cross-quadcopter platform for indoor-environments.

One contribution of the work was the design of a new printed-circuit-board (PCB) flight controller (called MARK3). Key features/capabilities are as follows:

(1) a Teensy 3.2 microcontroller with 168MHz overclock –used for communications, full-state estimation and inner-outer loop hierarchical rate-angle-speed-position control,

(2) an on-board MEMS inertial-measurement-unit (IMU) which includes an LSM303D (3DOF-accelerometer and magnetometer), an L3GD20 (3DOF-gyroscope) and a BMP180 (barometer) for attitude estimation (barometer/magnetometer not used),

(3) 6 pulse-width-modulator (PWM) output pins supports up to 6 rotors

(4) 8 PWM input pins support up to 8-channel 2.4 GHz transmitter/receiver for manual control,

(5) 2 5V servo extension outputs for other requirements (e.g. gimbals),

(6) 2 universal-asynchronous-receiver-transmitter (UART) serial ports - used by flight controller to process data from Xbee; can be used for accepting outer-loop position commands from NVIDIA TX2 (future work),

(7) 1 I2C-serial-protocol two-wire port for additional modules (used to read data from IMU at 400 Hz),

(8) a 20-pin port for Xbee telemetry module connection; permits Xbee transceiver on desktop PC to send position/attitude commands to Xbee transceiver on quadcopter.

The quadcopter platform consists of the new MARK3 PCB Flight Controller, an ATG-250 carbon-fiber frame (250 mm), a DJI Snail propulsion-system (brushless-three-phase-motor, electronic-speed-controller (ESC) and propeller), an HTC VIVE Tracker and RadioLink R9DS 9-Channel 2.4GHz Receiver. This platform is completely compatible with the HTC VIVE Tracking System (HVTS) which has 7ms latency, submillimeter accuracy and a much lower price compared to other millimeter-level tracking systems.

The thesis describes nonlinear and linear modeling of the quadcopter’s 6DOF rigid-body dynamics and brushless-motor-actuator dynamics. These are used for hierarchical-classical-control-law development near hover. The HVTS was used to demonstrate precision hover-control and path-following. Simulation and measured flight-data are shown to be similar. This work provides a foundation for future precision multi-quadcopter formation-flight-control.

This dissertation is focused on developing an algorithm to provide current state estimation and future state predictions for biomechanical human walking features. The goal is to develop a system which is capable of evaluating the current action a subject is taking while walking and then use this to predict the future states of biomechanical features.

This work focuses on the exploration and analysis of Interaction Primitives (Amor er al, 2014) and their relevance to biomechanical prediction for human walking. Built on the framework of Probabilistic Movement Primitives, Interaction Primitives utilize an EKF SLAM algorithm to localize and map a distribution over the weights of a set of basis functions. The prediction properties of Bayesian Interaction Primitives were utilized to predict real-time foot forces from a 9 degrees of freedom IMUs mounted to a subjects tibias. This method shows that real-time human biomechanical features can be predicted and have a promising link to real-time controls applications.

The purpose of this dissertation is to develop a design technique for fractional PID controllers to achieve a closed loop sensitivity bandwidth approximately equal to a desired bandwidth using frequency loop shaping techniques. This dissertation analyzes the effect of the order of a fractional integrator which is used as a target on loop shaping, on stability and performance robustness. A comparison between classical PID controllers and fractional PID controllers is presented. Case studies where fractional PID controllers have an advantage over classical PID controllers are discussed. A frequency-domain loop shaping algorithm is developed, extending past results from classical PID’s that have been successful in tuning controllers for a variety of practical systems.

This study was designed to create a more user-friendly experience in utilizing the M-Sorter package for spike sorting and detection. This was achieved through the creation of a Graphical User Interface or GUI. The GUI was created for the spike detection portion of sorting. Through the creation of the M-Sorter detection GUI, now novice programmers can run the detector process. Additionally, the parameters are easily altered which will greatly decrease the time it takes to enter data and eliminate mistakes users may make in data entry.

In this paper, it is determined that learning retention decreases with age and there is a linear rate of decrease. In this study, four male Long-Evans Rats were used. The rats were each trained in 4 different tasks throughout their lifetime, using a food reward as motivation to work. Rats were said to have learned a task at the age when they received the highest accuracy during a task. A regression of learning retention was created for the set of studied rats: Learning Retention = 112.9 \u2014 0.085919 x (Age at End of Task), indicating that learning retention decreases at a linear rate, although rats have different rates of decrease of learning retention. The presence of behavioral training was determined not to have a positive impact on this rate. In behavioral studies, there were statistically significant differences between timid/outgoing and large ball ability between W12 and Z12. Rat W12 had overall better learning retention and also was more compliant, did not resist being picked up and traveled more frequently at high speeds (in the large ball) than Z12. Further potential studies include implanting an electrode into the frontal cortex in order to compare neuro feedback with learning retention, and using human subjects to find the rate of decrease in learning retention. The implication of this study, if also true for human subjects, is that older persons may need enhanced training or additional refresher training in order to retain information that is learned at a later age.