Novelty detection in autonomous systems
We have shown how novelty detection algorithms for monitoring, reconnaissance, and search-and-rescue tasks can be made more robust through highly adaptive neural networks. Nonetheless, the robustness depends on the operational environments and type of task, which has implications for how novelty detection is implemented in real-world situations as well as for the training of human co-workers.
Project team: Lenka Pitonakova and Seth Bullock
Finding novelty
Robots such as unmanned aerial vehicles (UAVs) are commonly used in surveillance tasks. However, detecting search objects can be extremely challenging, particularly for new types of objects.
An alternative approach is to allow robots to detect change or novelty in their environment. In principle, this offers greater sensitivity to novel events.
Automated systems that can detect novelty in themselves may also offer greater reliability through identifying and reporting changes in the behaviour of the system itself.
However, the performance limits of automated novelty detection are currently unknown. We tackled this by investigating novelty detection in different environments with different types of change to develop an algorithm that adapts to the complexity of the situation.
Our approach
The trade-off between fidelity and robustness is key in novelty detection, as greater fidelity allows smaller features to be detected but the system will become more sensitive to noise.
We developed an adaptive neural network that tunes the level of fidelity for individual features, either increasing or decreasing definition when there is insufficient or too much detail.
Using this model, we simulated robots searching for novelty in two environments: a room and a corridor. Each included elements such as doors and boxes that a robot could be trained to recognise from its camera images, although the corridor had a more diverse range of features. We created novelty in the system by either introducing or removing features over time.
Novel results
Our adaptive detection algorithm outperformed all alternative methods of novelty detection and provided insight into the robustness of the approach. Our key findings were:
- Novelty detection senses the appearance of an object more easily than a disappearance;
- Novel events in the simpler room model was detected more easily than in the corridor model;
- Single novel events within a simulation could be detected with great reliability but events in a series of novelties could go undetected. We solved this simply by linking the model to the robot’s location coordinates.
The human element
The environment and task dependence of automated novelty detection carries a cautionary note. Human co-workers will tend to place greater trust in a system that becomes more robust. The danger is that complacency sets in, resulting in false interpretations of the monitored environment.
It is vital, therefore, that scepticism of any novelty detection technology becomes an important part of human training and practice.
Imperfect robustness may also by mitigated by tuning the fidelity-robustness trade-off towards false positives rather than false negatives to reduce the chance of missing real events.
Future trends
Novelty detection has the potential to make reconnaissance, surveillance, and search-and-rescue tasks more efficient. It could also allow systems to self-monitor at various levels, from that of sensors and motors to individual robots and through to systems involving multiple assets and people.
Our work revealing the limitations of a general-purpose novelty detection system shows that end users must ensure any training environment closely matches the operational environment and that human operatives receive training appropriate to the robustness of the technology. Indeed, understanding the limitations of novelty detection technology will be key to realising its potential.