Knowing What You Know: Calibrating Dialogue Belief State Distributions via Ensembles

Published in Findings of the Association for Computational Linguistics: EMNLP 2020, 2020

The ability to accurately track what happens during a conversation is essential for the performance of a dialogue system. Current state-of-the-art multi-domain dialogue state trackers achieve just over 55% accuracy on the current go-to benchmark, which means that in almost every second dialogue turn they place full confidence in an incorrect dialogue state. Belief trackers, on the other hand, maintain a distribution over possible dialogue states. However, they lack in performance compared to dialogue state trackers, and do not produce well calibrated distributions. In this work we present state-of-the-art performance in calibration for multi-domain dialogue belief trackers using a calibrated ensemble of models. Our resulting dialogue belief tracker also outperforms previous dialogue belief tracking models in terms of accuracy.

Download here

Adaptable Conversational Machines

Published in Adaptable Conversational Machines. AI Magazine, Vol. 41 No. 3: Fall 2020, 2020

In recent years we have witnessed a surge in machine learning methods that provide machines with conversational abilities. Most notably, neural-network–based systems have set the state of the art for difficult tasks such as speech recognition, semantic understanding, dialogue management, language generation, and speech synthesis. Still, unlike for the ancient game of Go for instance, we are far from achieving human-level performance in dialogue. The reasons for this are numerous. One property of human–human dialogue that stands out is the infinite number of possibilities of expressing oneself during the conversation, even when the topic of the conversation is restricted. A typical solution to this problem was scaling-up the data. The most prominent mantra in speech and language technology has been “There is no data like more data.” However, the researchers now are focused on building smarter algorithms — algorithms that can learn efficiently from just a few examples. This is an intrinsic property of human behavior: an average human sees during their lifetime a fraction of data that we nowadays present to machines. A human can even have an intuition about a solution before ever experiencing an example solution. The human-inspired ability to adapt may just be one of the keys in pushing dialogue systems toward human performance. This article reviews advancements in dialogue systems research with a focus on the adaptation methods for dialogue modeling, and ventures to have a glance at the future of research on adaptable conversational machines.

Recommended citation: Lubis, N. ., Heck, M., van Niekerk, C., & Gasic, M. (2020). Adaptable Conversational Machines. AI Magazine, 41(3), 28-44.

Multiscale spatial modeling with applications in image analysis

Published in Dissertation (MSc)--University of Pretoria., 2018

Computer vision is a very important research area and is continuously growing. One of the prevalent research areas in computer vision is image matching. In image matching there are two main components, namely feature detection and feature matching. The aim of this this study is to determine whether Direct Sampling can be used for feature matching, and also if the combination of Direct Sampling and the Discrete Pulse Transform feature detector can be a successful image matching tool. In feature detection there are many strong methods including convolutional neural networks and scale-space models such as SIFT and SURF, which are very well-known feature detection algorithms. In this work we utilize another scale-space decomposition tool called the Discrete Pulse Transform (DPT). We particularly use the DPT decomposition to enable significant feature detection. We then concentrate on using the Direct Sampling algorithm, a stochastic spatial simulation algorithm, for modelling and matching of features. We do not consider convolutional neural networks or SIFT or SURF for texture matching in this work, this is because we particularly focus on the use of spatial statistics in image matching. We finally propose a novel multiscale spatial statistics feature detection and matching algorithm which combines the DPT feature detection with Direct Sampling for feature matching, specifically for texture classes of images. The performance of the proposed method is tested by comparing the distances obtained from the proposed algorithm between different texture images. We see that this proposed novel multiscale spatial modelling approach to feature matching with the focus on textures performs well at discriminating between difficult to discriminate between textures.

Download here