Image processing based control of mobile robotics

Toward the ambitious long-term goal of a fleet of cooperating Flexible Autonomous Machines operating in an uncertain Environment (FAME), this thesis addresses various control objectives for ground vehicles.

There are two main objectives within this thesis, first is the use of visual information to control a Differential-Drive Thunder Tumbler (DDTT) mobile robot and second is the solution to a minimum time optimal control problem for the robot around a racetrack.

One method to do the first objective is by using the Position Based Visual Servoing (PBVS) approach in which a camera looks at a target and the position of the target with respect to the camera is estimated; once this is done the robot can drive towards a desired position (x_ref, z_ref). Another method is called Image Based Visual Servoing (IBVS), in which the pixel coordinates (u,v) of markers/dots placed on an object are driven towards the desired pixel coordinates (u_ref, v_ref) of the corresponding markers.

By doing this, the mobile robot gets closer to a desired pose (x_ref, z_ref, theta_ref).

For the second objective, a camera-based and noncamera-based (v,theta) cruise-control systems are used for the solution of the minimum time problem. To set up the minimum time problem, optimal control theory is used. Then a direct method is implemented by discretizing states and controls of the system. Finally, the solution is obtained by modeling the problem in AMPL and submitting to the nonlinear optimization solver KNITRO. Simulation and experimental results are presented.

The DDTT-vehicle used within this thesis has different components as summarized below:

(1) magnetic wheel-encoders/IMU for inner-loop speed-control and outer-loop directional control,

(2) Arduino Uno microcontroller-board for encoder-based inner-loop speed-control and encoder-IMU-based outer-loop cruise-directional-control,

(3) Arduino motor-shield for inner-loop speed-control,

(4) Raspberry Pi II computer-board for outer-loop vision-based cruise-position-directional-control,

(5) Raspberry Pi 5MP camera for outer-loop cruise-position-directional control.

Hardware demonstrations shown in this thesis are summarized: (1) PBVS without pan camera, (2) PBVS with pan camera, (3) IBVS with 1 marker/dot, (4) IBVS with 2 markers, (5) IBVS with 3 markers, (6) camera and (7) noncamera-based (v,theta) cruise control system for the minimum time problem.