Uncertainty Estimation, Management, and Utilisation in Human-Computer Dialogue

In the rapidly evolving field of human-computer interaction, there is an increasing demand for effective and reliable dialogue systems, computer programs engineered to converse with humans. However, these systems often fall short in unpredictable or ambiguous scenarios, a problem attributed to the absence of a comprehensive model for handling uncertainty. This limitation impacts the ability to communicate effectively with human users, thereby diminishing user experience and trust. Uncertainty is a fundamental aspect of human cognition and everyday decision-making processes, serving as both an obstacle and an opportunity in our constant pursuit of knowledge and effective communication. Despite its important role, uncertainty is underrepresented in the development of machine learning models for dialogue systems.

To address these gaps, this thesis focuses on integrating and leveraging uncertainty within taskoriented dialogue systems, systems designed to assist users in accomplishing specific tasks. With the aim of achieving human-level interactive capabilities, we make three substantial contributions to this area. First, we enhance the system’s language understanding component, improving its accuracy in evaluating the certainty of its predictions. Secondly, we introduce SetSUMBT (Set Similarity based Slot Utterance Matching Belief Tracker), a model designed to capture various facets of uncertainty, bolstering the robustness and adaptability of the dialogue policy models responsible for generating system responses, as validated through simulated and real-user interactions. Thirdly, we present CAMELL (Confidence-based Acquisition Model for Efficient self-supervised active Learning with Label validation), an innovative framework which minimises the reliance of models on labelled data. By incorporating elements of self-supervision, where models learn from their own predictions, and label validation, CAMELL automates the rectification of unreliable human annotations, a feature with extensive applicability in various machine learning domains.

Incorporating insights from psychological theories on human uncertainty management, this thesis emphasises the importance of integrating such insights into machine learning models for dialogue. Our methods will advance the field by introducing more reliable, robust, and effective dialogue systems that better handle uncertainties, ultimately enhancing the quality of human-computer interaction. Furthermore, this work challenges current limitations associated with data deficiencies, offering a data-driven approach for improving dataset quality, thereby paving the way for future research in machine learning and human-computer interaction.

Download thesis here