Carel van Niekerk
About Me
I am a Postdoctoral Researcher in the Dialogue Systems and Machine Learning group at Heinrich Heine University, Düsseldorf, under Prof. Milica Gašić. My research is currently part of the ‘Lamarr Fellow Network Ramp Up’ project, with a special interest in understanding uncertainty in Large Language Models (LLMs). This follows my successful completion of a PhD, which I received with magna cum laude.
I have 10 years of experience across academia and industry, with a focus on machine learning, deep learning, and natural language processing. I have over five years of experience conducting cutting-edge AI research and have spent a year in AI consulting, applying state-of-the-art methods to real-world problems.
Previously, I co-authored the 2020 SIGDial best paper “TripPy”, and before that, I worked as an AI consultant at NGA Risksecure. My academic background includes a Master’s degree in Mathematical Statistics from the University of Pretoria, where I graduated with distinction and served as a teaching and research assistant. My goal is to improve dialogue systems by making them smarter and more reliable, through a deeper understanding of LLMs.
My interests are in Data and AI, focusing on designing and implementing AI and genAI solutions. I have designed, developed and deployed genAI applications with real-world impact, including the implementation of AI agents on the Google Cloud Platform to automate human evaluation tasks. My research includes work with both API-based and on-premise large language models (LLMs), with a particular focus on reinforcement learning for LLM fine-tuning.
I have published at leading venues such as EMNLP, ACL and IEEE, with contributions centred on generative modelling and trustworthy AI. I also have experience in mentorship, having successfully supervised several Master’s and Bachelor’s theses.
Highlights
- 1 October 2025 - Our paper, Less is More: Local Intrinsic Dimensions of Contextual Language Models, has been accepted for presentation at NeurIPS 2025. In this work, we introduce a novel perspective based on the geometric properties of contextual latent embeddings to study the effects of training and fine-tuning in large language models (LLMs). We show that local dimensions provide insights into a model’s training dynamics and generalization ability, predicting phenomena such as training exhaustion, overfitting, and grokking across various tasks. Our findings offer practical heuristics for model configuration and contribute to the discourse on LLM interpretability, adaptability, and generalizability by bridging intrinsic model mechanisms with geometric properties in embeddings.
- 29 July 2025 - We have released a preprint of our paper, Post-Training Large Language Models via Reinforcement Learning from Self-Feedback. In this work, we introduce Reinforcement Learning from Self-Feedback (RLSF), a novel post-training approach that leverages a model’s own confidence as an intrinsic reward. This method enhances both the calibration and reasoning capabilities of Large Language Models (LLMs) without the need for human labels or external rewards, marking a significant advancement in LLM post-training techniques.
- 1 July 2025 - Our paper, A Confidence-based Acquisition Model for Self-supervised Active Learning and Label Correction, has been published in Transactions of the Association for Computational Linguistics (TACL) and will be presented at ACL 2025 in Vienna. In this work, we introduce CAMEL, a novel active learning framework designed for sequential multi-output tasks, which significantly enhances efficiency and data quality through partial expert labeling and self-supervision.
- 9 April 2024 - Today I recieved my Ph.D. with magna cum laude. My thesis, titled “Uncertainty Estimation, Management, and Utilisation in Human-Computer Dialogue”, explores the role of uncertainty in Language Understanding and its implications for dialogue systems. I am grateful for the support of my advisor, Prof. Milica Gašić, and the members of the Dialogue Systems and Machine Learning group at Heinrich Heine University.