Human-robot interactions can often be formulated as general-sum differential games where the equilibrial policies are governed by Hamilton-Jacobi-Isaacs (HJI) equations. Solving HJI PDEs faces the curse of dimensionality (CoD). While physics-informed neural networks (PINNs) alleviate CoD in solving PDEs with smooth solutions, they fall short in learning discontinuous solutions due to their sampling nature. This causes PINNs to have poor safety performance when they are applied to approximate values that are discontinuous due to state constraints. This dissertation aims to improve the safety performance of PINN-based value and policy models. The first contribution of the dissertation is to develop learning methods to approximate discontinuous values. Specifically, three solutions are developed: (1) hybrid learning uses both supervisory and PDE losses, (2) value-hardening solves HJIs with increasing Lipschitz constant on the constraint violation penalty, and (3) the epigraphical technique lifts the value to a higher-dimensional state space where it becomes continuous. Evaluations through 5D and 9D vehicle and 13D drone simulations reveal that the hybrid method outperforms others in terms of generalization and safety performance. The second contribution is a learning-theoretical analysis of PINN for value and policy approximation. Specifically, by extending the neural tangent kernel (NTK) framework, this dissertation explores why the choice of activation function significantly affects the PINN generalization performance, and why the inclusion of supervisory costate data improves the safety performance. The last contribution is a series of extensions of the hybrid PINN method to address real-time parameter estimation problems in incomplete-information games. Specifically, a Pontryagin-mode PINN is developed to avoid costly computation for supervisory data. The key idea is the introduction of a costate loss, which is cheap to compute yet effectively enables the learning of important value changes and policies in space-time. Building upon this, a Pontryagin-mode neural operator is developed to achieve state-of-the-art (SOTA) safety performance across a set of differential games with parametric state constraints. This dissertation demonstrates the utility of the resultant neural operator in estimating player constraint parameters during incomplete-information games.

The Endoscopic Submucosal Dissection (ESD) method is increasingly becoming the method of choice for surgeons attempting to remove precancerous and early-stage cancerous lesions in the lining of the Gastrointestinal (GI) tract. Being an endoscopic procedure, it is less invasive than most other procedures used for tumor removal. However, this procedure has a steep learning curve and a high number of surgical complications. The primary reason for this is the limited ability of the surgeon to retract mucosal (stomach lining) tissue while they dissect under it. Unlike in traditional surgery, the surgeon lacks a second hand to leverage tissue during dissection in endoscopic procedures. This study proposed the deployment of an endoscopic clip to the surface of the lesion. The clip had a permanent magnet connected to it. In addition, a large permanent external magnet mounted to the end-effector of a robotic arm was positioned above the magnetic clip to pull the internal magnet and retract tissue. Magnetic Force simulations were conducted in the design processes for the magnets to determine whether sufficient force for tissue retraction was being achieved. The use of fiber optic shape sensors to track and localize the internal magnet was also explored. Experimental validations of the external and internal magnet designs as well as tracking of the internal magnet were performed in surgical trials on ex-vivo and live porcine models. Compared to traditional ESD, the use of magnetic retraction in ESD significantly improved tissue exposure for dissection, decreased the required time for the dissection stage of the ESD procedure, and reduced the incidence of surgical complications. Therefore, this technology holds substantial potential for enhancing ESD procedures, advancing the non-invasive treatment of colorectal cancer, and potentially improving patient outcomes significantly.

In this thesis, I investigate a subset of reinforcement learning (RL) tasks where the objective for the agent is to achieve temporally extended goals. A common approach, in this setting, is to represent the tasks using deterministic finite automata (DFA) and integrate them in the state space of the RL algorithms, yet such representations often disregard causal knowledge pertinent to the environment. To address this limitation, I introduce the Temporal-Logic-based Causal Diagram (TL-CD) in RL.TL-CD encapsulates temporal causal relationships among diverse environmental properties. We leverage the TL-CD to devise an RL algorithm that significantly reduces environment exploration requirements. By synergizing TL-CD with task-specific DFAs, I identify scenarios wherein the agent can efficiently determine expected rewards early during the exploration phases. Through a series of case studies, I empirically demonstrate the advantages of TL-CDs, particularly highlighting the accelerated convergence of the algorithm towards an optimal policy facilitated by diminished exploration of the environment.

Animals have always been a source of inspiration for real-life problems. The octopus is one such animal that has a lot of untapped potential. The octopus’s arm is without solid joints or bone structure and despite this it can achieve many complicated movements with virtually infinite degrees of freedom. This ability is made possible through the unique morphology of the arm. The octopus’s arm is divided into transverse, longitudinal, oblique, and circular muscle groups and each one has a unique muscle fiber orientation. The octopus’s arm is classified as a hydrostat because it maintains a constant volume while contracting with the help of its different muscle groups. These muscle groups allow elongation, shortening, bending, and twisting of the arm when they work in combination with each other. To confirm the role of transverse and longitudinal muscle groups, an electromyography (EMG) recording of these muscle groups was performed while an amputated arm of an Octopus bimaculoides was stimulated with an electrical signal to induce movement. Statistical analysis was performed on these results to confirm the roles of each muscle group quantitatively. Octopus arm morphology was previously assumed to be uniform along the arm. Through a magnetic resonance imaging (MRI) study at the proximal, middle, and distal sections of the arm this notion was disproven, and a new pattern was discovered. Drawing inspiration from this finding and previous octopus arm prototypes, 4 bio-inspired designs were conceived and tested in finite element analysis (FEA) simulations. Four tests in elongation, shortening, bending, and transverse-assisted bending movements were performed on all designs to compare each design’s performance. The findings in this study have applications in engineering and soft robotics fields for use cases such as, handling fragile objects, minimally invasive surgeries, difficult-to-access areas that require squeezing through small holes, and other novel cases.

This thesis presents a study on the user adaptive variable impedance control of a wearable ankle robot for robot-aided rehabilitation with a primary focus on enhancing accuracy and speed. The controller adjusts the impedance parameters based on the user's kinematic data to provide personalized assistance. Bayesian optimization is employed to minimize an objective function formulated from the user's kinematic data to adapt the impedance parameters per user, thereby enhancing speed and accuracy. Gaussian process is used as a surrogate model for optimization to account for uncertainties and outliers inherent to human experiments. Student-t process based outlier detection is utilized to enhance optimization robustness and accuracy. The efficacy of the optimization is evaluated based on measures of speed, accuracy, and effort, and compared with an untuned variable impedance controller during 2D curved trajectory following tasks. User effort was measured based on muscle activation data from the tibialis anterior, peroneus longus, soleus, and gastrocnemius muscles. The optimized controller was evaluated on 15 healthy subjects and demonstrated an average increase in speed of 9.85% and a decrease in deviation from the ideal trajectory of 7.57%, compared to an unoptimized variable impedance controller. The strategy also reduced the time to complete tasks by 6.57%, while maintaining a similar level of user effort.

Ferrofluidic microrobots have emerged as promising tools for minimally invasive medical procedures, leveraging their unique properties to navigate through complex fluids and reach otherwise inaccessible regions of the human body, thereby enabling new applications in areas such as targeted drug delivery, tissue engineering, and diagnostics. This dissertation develops a model-predictive controller for the external magnetic manipulation of ferrofluid microrobots. Several experiments are performed to illustrate the adaptability and generalizability of the control algorithm to changes in system parameters, including the three-dimensional reference trajectory, the velocity of the workspace fluid, and the size, orientation, deformation, and velocity of the microrobotic droplet. A linear time-invariant control system governing the dynamics of locomotion is derived and used as the constraints of a least squares optimal control algorithm to minimize the projected error between the actual trajectory and the desired trajectory of the microrobot. The optimal control problem is implemented after time discretization using quadratic programming. In addition to demonstrating generalizability and adaptability, the accuracy of the control algorithm is analyzed for several different types of experiments. The experiments are performed in a workspace with a static surrounding fluid and extended to a workspace with fluid flowing through it. The results suggest that the proposed control algorithm could enable new capabilities for ferrofluidic microrobots, opening up new opportunities for applications in minimally invasive medical procedures, lab-on-a-chip, and microfluidics.

This thesis considers the problem of multi-robot task allocation with inter-agent distance constraints, e.g., due to the presence of physical tethers or communication requirements, that must be satisfied at all times. Specifically, three optimization-based formulations are explored: (i) a “Naive Method” that leverages the classical multiple traveling salesman (mTSP) formulation to find solutions that are then filtered out when the inter-agent distance constraints are violated, (ii) a “Timed Method” thatconstructs a new formulation that explicitly accounts for robot timings, including the inter-agent distance constraints, and (iii) an “Improved Naive Method” that reformulates the Naive Method with a novel graph-traversal algorithm to produce tours that, unlike the Naive Method, allow backtracking and also introduces a more systematic approach to filter out solutions that violate inter-agent distance constraints. The effectiveness of the approaches to return task allocations that satisfy the constraints are demonstrated and compared in simulation experiments.

Advanced driving assistance systems (ADAS) are one of the latest automotive technologies for improving vehicle safety. An efficient method to ensure vehicle safety is to limit vehicle states always within a predefined stability region. Hence, this thesis aims at designing a model predictive control (MPC) with non-overshooting constraints that always confine vehicle states in a predefined lateral stability region. To consider the feasibility and stability of MPC, terminal cost and constraints are investigated to guarantee the stability and recursive feasibility of the proposed non-overshooting MPC. The proposed non-overshooting MPC is first verified by using numerical examples of linear and nonlinear systems. Finally, the non-overshooting MPC is applied to guarantee vehicle lateral stability based on a nonlinear vehicle model for a cornering maneuver. The simulation results are presented and discussed through co-simulation of CarSim® and MATLAB/Simulink.

Magnetic liquids called ferrofluids have been used in applications ranging from audio speaker cooling and rotary pressure seals to retinal detachment surgery and implantable artificial glaucoma valves. Recently, ferrofluids have been investigated as a material for use in magnetically controllable liquid droplet robotics. Liquid droplet robotics is an emerging technology that aims to apply control theory to manipulate fluid droplets as robotic agents to perform a wide range of tasks. Furthermore, magnetically controlled micro-robotics is another popular area of study where manipulating a magnetic field allows for the control of magnetized micro-robots. Both of these emerging fields have potential for impact toward medical applications: liquid characteristics such as being able to dissolve various compounds, be injected via a needle, and the potential for the human body to automatically filter and remove a liquid droplet robot, make liquid droplet robots advantageous for medical applications; while the ability to remotely control the torques and forces on an untethered microrobot via modulating the magnetic field and gradient is also highly advantageous. The research described in this dissertation explores applications and methods for the electromagnetic control of ferrofluid droplet robots. First, basic electrical components built from fluidic channels containing ferrofluid are made remotely tunable via the placement of ferrofluid within the channel. Second, a ferrofluid droplet is shown to be fully controllable in position, stretch direction, and stretch length in two dimensions using proportional-integral-derivative (PID) controllers. Third, control of a ferrofluid’s position, stretch direction, and stretch length is extended to three dimensions, and control gains are optimized via a Bayesian optimization process to achieve higher accuracy. Finally, magnetic control of both single and multiple ferrofluid droplets in two dimensions is investigated via a visual model predictive control approach based on machine learning. These achievements take both liquid droplet robotics and magnetic micro-robotics fields several steps closer toward real-world medical applications such as embedded soft electronic health monitors, liquid-droplet-robot-based drug delivery, and automated magnetically actuated surgeries.

The unparalleled motion and manipulation abilities of an octopus have intrigued engineers and biologists for many years. How can an octopus having no bones transform its arms from a soft state to a one stiff enough to catch and even kill prey? The octopus arm is a muscular hydrostat that enables these manipulations in and through its arm. The arm is a tightly packed array of muscle groups namely longitudinal, transverse and oblique. The orientation of these muscle fibers aids the octopus in achieving core movements like shortening, bending, twisting and elongation as hypothesized previously. Through localized electromyography (EMG) recordings of the longitudinal and transverse muscles of Octopus bimaculoides quantitatively the roles of these muscle layers will be confirmed. Five EMG electrode probes were inserted into the longitudinal and transverse muscle layers of an amputated octopus arm. One into the axial nerve cord to electrically stimulate the arm for movements. The experiments were conducted with the amputated arm submerged in sea water with surrounded cameras to record the movement, all housed in a Faraday cage. The findings of this research could possibly lead to the development of soft actuators built out of soft materials for applications in minimally invasive surgery, search-and-rescue operations, and wearable prosthetics.