Context Integration for Reliable Anomaly Detection from Imagery Data for Supporting Civil Infrastructure Operation and Maintenance

Imagery data has become important for civil infrastructure operation and

maintenance because imagery data can capture detailed visual information with high

frequencies. Computer vision can be useful for acquiring spatiotemporal details to

support the timely maintenance of critical civil infrastructures that serve society. Some

examples include: irrigation canals need to maintain the leaking sections to avoid water

loss; project engineers need to identify the deviating parts of the workflow to have the

project finished on time and within budget; detecting abnormal behaviors of air traffic

controllers is necessary to reduce operational errors and avoid air traffic accidents.

Identifying the outliers of the civil infrastructure can help engineers focus on targeted

areas. However, large amounts of imagery data bring the difficulty of information

overloading. Anomaly detection combined with contextual knowledge could help address

such information overloading to support the operation and maintenance of civil

infrastructures.

Some challenges make such identification of anomalies difficult. The first challenge is

that diverse large civil infrastructures span among various geospatial environments so

that previous algorithms cannot handle anomaly detection of civil infrastructures in

different environments. The second challenge is that the crowded and rapidly changing

workspaces can cause difficulties for the reliable detection of deviating parts of the

workflow. The third challenge is that limited studies examined how to detect abnormal

behaviors for diverse people in a real-time and non-intrusive manner. Using video andii

relevant data sources (e.g., biometric and communication data) could be promising but

still need a baseline of normal behaviors for outlier detection.

This dissertation presents an anomaly detection framework that uses contextual

knowledge, contextual information, and contextual data for filtering visual information

extracted by computer vision techniques (ADCV) to address the challenges described

above. The framework categorizes the anomaly detection of civil infrastructures into two

categories: with and without a baseline of normal events. The author uses three case

studies to illustrate how the developed approaches can address ADCV challenges in

different categories of anomaly detection. Detailed data collection and experiments

validate the developed ADCV approaches.