Publications

2025

1. 

   HAIFAI: Human-AI Interaction for Mental Face Reconstruction

   Florian Strohm, Mihai Bâce, and Andreas Bulling

   ACM Transactions on Interactive Intelligent Systems (TiiS), 2025

   Abs DOI PDF

   We present HAIFAI — a novel two-stage system where humans and AI interact to tackle the challenging task of reconstructing a visual representation of a face that exists only in a person’s mind. In the first stage, users iteratively rank images our reconstruction system presents based on their resemblance to a mental image. These rankings, in turn, allow the system to extract relevant image features, fuse them into a unified feature vector and use a generative model to produce an initial reconstruction of the mental image. The second stage leverages an existing face editing method, allowing users to manually refine and further improve this reconstruction using an easy-to-use slider interface for face shape manipulation. To avoid the need for tedious human data collection for training the reconstruction system, we introduce a computational user model of human ranking behaviour. For this, we collected a small face ranking dataset through an online crowd-sourcing study containing data from 275 participants. We evaluate HAIFAI and an ablated version in a 12-participant user study and demonstrate that our approach outperforms the previous state of the art regarding reconstruction quality, usability, perceived workload and reconstruction speed. We further validate the reconstructions in a subsequent face ranking study with 18 participants and show that HAIFAI achieves a new state-of-the-art identification rate of 60.6%. These findings represent a significant advancement towards developing new interactive intelligent systems capable of reliably and effortlessly reconstructing a user’s mental image.

2. DiffGaze: A Diffusion Model for Modelling Fine-grained Human Gaze Behaviour on 360° Images

   Chuhan Jiao, Yao Wang, Guanhua Zhang, Mihai Bâce, Zhiming Hu, and Andreas Bulling

   ACM Transactions on Interactive Intelligent Systems (TiiS), Oct 2025

   Abs DOI PDF

   Modelling human gaze behaviour on 360 (^circ) images is important for various human-computer interaction applications. However, existing methods are limited to predicting discrete fixation sequences or aggregated saliency maps, thereby neglecting fine-grained gaze behaviour such as saccadic eye movements that can be captured by commercial eye-trackers. We introduce a more challenging task—fine-grained gaze sequence generation. This task aims to generate eye-tracker-like gaze data for given stimuli. We propose DiffGaze, a diffusion-based method for generating realistic and diverse fine-grained human gaze sequences conditioned on 360 (^circ) images. We evaluate DiffGaze on two 360 (^circ) image benchmarks for fine-grained gaze sequence generation as well as two downstream tasks, scanpath prediction and saliency prediction. Our evaluations show that DiffGaze outperforms the fine-grained gaze generation baselines in all tasks on both benchmarks. We also report a 21-participant survey study showing that our method generates gaze sequences that are indistinguishable from real human sequences. Taken together, our evaluations not only demonstrate the effectiveness of DiffGaze but also point towards a new generation of methods that faithfully model the rich spatial and temporal nature of natural human gaze behaviour.

2024

1. SeFFeC: Semantic Facial Feature Control for Fine-grained Face Editing

   Florian Strohm, Mihai Bâce, Markus Kaltenecker, and Andreas Bulling

   Oct 2024

   Abs DOI

   We propose Semantic Facial Feature Control (SeFFeC) - a novel method for fine-grained face shape editing. Our method enables the manipulation of human-understandable, semantic face features, such as nose length or mouth width, which are defined by different groups of facial landmarks. In contrast to existing methods, the use of facial landmarks enables precise measurement of the facial features, which then enables training SeFFeC without any manually annotated labels. SeFFeC consists of a transformer-based encoder network that takes a latent vector of a pre-trained generative model and a facial feature embedding as input, and learns to modify the latent vector to perform the desired face edit operation. To ensure that the desired feature measurement is changed towards the target value without altering uncorrelated features, we introduced a novel semantic face feature loss. Qualitative and quantitative results show that SeFFeC enables precise and fine-grained control of 23 facial features, some of which could not previously be controlled by other methods, without requiring manual annotations. Unlike existing methods, SeFFeC also provides deterministic control over the exact values of the facial features and more localised and disentangled face edits.

2. UP-FacE: User-predictable Fine-grained Face Shape Editing

   Florian Strohm, Mihai Bâce, and Andreas Bulling

   Oct 2024
3. Int-HRL: Towards Intention-based Hierarchical Reinforcement Learning

   Anna Penzkofer, Simon Schaefer, Florian Strohm, Mihai Bâce, Stefan Leutenegger, and Andreas Bulling

   Neural Computing and Applications (NCAA), Oct 2024

   DOI
4. Learning User Embeddings from Human Gaze for Personalised Saliency Prediction

   Florian Strohm, Mihai Bâce, and Andreas Bulling

   Proc. ACM on Human-Computer Interaction (PACM HCI), Oct 2024

   DOI
5. VisRecall++: Analysing and Predicting Visualisation Recallability from Gaze Behaviour

   Yao Wang, Yue Jiang, Zhiming Hu, Constantin Ruhdorfer, Mihai Bâce, and Andreas Bulling

   Proc. ACM on Human-Computer Interaction (PACM HCI), Oct 2024

   DOI
6. SalChartQA: Question-driven Saliency on Information Visualisations

   Yao Wang, Weitian Wang, Abdullah Abdelhafez, Mayar Elfares, Zhiming Hu, Mihai Bâce, and Andreas Bulling

   In Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), Oct 2024

   DOI
7. Mouse2Vec: Learning Reusable Semantic Representations of Mouse Behaviour

   Guanhua Zhang, Zhiming Hu, Mihai Bâce, and Andreas Bulling

   In Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), Oct 2024

   DOI
8. Saliency3D: a 3D Saliency Dataset Collected on Screen

   Yao Wang, Qi Dai, Mihai Bâce, Karsten Klein, and Andreas Bulling

   In Proc. ACM International Symposium on Eye Tracking Research and Applications (ETRA), Oct 2024

   DOI

2023

1. Usable and Fast Interactive Mental Face Reconstruction

   Florian Strohm, Mihai Bâce, and Andreas Bulling

   In Proc. ACM Symposium on User Interface Software and Technology (UIST), Oct 2023

   DOI
2. SUPREYES: SUPer Resolution for EYES Using Implicit Neural Representation Learning

   Chuhan Jiao, Zhiming Hu, Mihai Bâce, and Andreas Bulling

   In Proc. ACM Symposium on User Interface Software and Technology (UIST), Oct 2023

   DOI
3. Int-HRL: Towards Intention-based Hierarchical Reinforcement Learning

   Anna Penzkofer, Simon Schaefer, Florian Strohm, Mihai Bâce, Stefan Leutenegger, and Andreas Bulling

   In Proc. Adaptive and Learning Agents Workshop (ALA), Oct 2023
4. Multimodal Integration of Human-Like Attention in Visual Question Answering

   Ekta Sood, Fabian Kögel, Philipp Müller, Dominike Thomas, Mihai Bâce, and Andreas Bulling

   In Proc. Workshop on Gaze Estimation and Prediction in the Wild (GAZE), CVPRW, Oct 2023
5. Scanpath Prediction on Information Visualisations

   Yao Wang, Mihai Bâce, and Andreas Bulling

   IEEE Transactions on Visualization and Computer Graphics (TVCG), Oct 2023

   DOI
6. Exploring Natural Language Processing Methods for Interactive Behaviour Modelling

   Guanhua Zhang, Matteo Bortoletto, Zhiming Hu, Lei Shi, Mihai Bâce, and Andreas Bulling

   In Proc. IFIP TC13 Conference on Human-Computer Interaction (INTERACT), Oct 2023

2022

1. VisRecall: Quantifying Information Visualisation Recallability via Question Answering

   Yao Wang, Chuhan Jiao, Mihai Bâce, and Andreas Bulling

   IEEE Transactions on Visualization and Computer Graphics (TVCG), Oct 2022

   DOI
2. Impact of Gaze Uncertainty on AOIs in Information Visualisations

   Yao Wang, Maurice Koch, Mihai Bâce, Daniel Weiskopf, and Andreas Bulling

   In ETRA Workshop on Eye Tracking and Visualization (ETVIS), Oct 2022

   DOI
3. Facial Composite Generation with Iterative Human Feedback

   Florian Strohm, Ekta Sood, Dominike Thomas, Mihai Bâce, and Andreas Bulling

   In Proceedings of the NeurIPS Workshop Gaze Meets ML (GMML), Oct 2022
4. Designing for Noticeability: The Impact of Visual Importance on Desktop Notifications

   Philipp Müller, Sander Staal, Mihai Bâce, and Andreas Bulling

   In Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), Oct 2022

   DOI
5. PrivacyScout: Assessing Vulnerability to Shoulder Surfing on Mobile Devices

   Mihai Bâce, Alia Saad, Mohamed Khamis, Stefan Schneegass, and Andreas Bulling

   In Proc. on Privacy Enhancing Technologies (PETs), Oct 2022

   DOI
6. Predicting Next Actions and Latent Intents during Text Formatting

   Guanhua Zhang, Susanne Hindennach, Jan Leusmann, Felix Bühler, Benedict Steuerlein, Sven Mayer, Mihai Bâce, and Andreas Bulling

   In Proceedings of the CHI Workshop Computational Approaches for Understanding, Generating, and Adapting User Interfaces, Oct 2022
7. Neuro-Symbolic Visual Dialog

   Adnen Abdessaied, Mihai Bâce, and Andreas Bulling

   In Proceedings of the 29th International Conference on Computational Linguistics (COLING), Oct 2022

2021

1. Neural Photofit: Gaze-based Mental Image Reconstruction

   Florian Strohm, Ekta Sood, Sven Mayer, Philipp Müller, Mihai Bâce, and Andreas Bulling

   In Proc. IEEE International Conference on Computer Vision (ICCV), Oct 2021

   DOI

2020

1. How far are we from quantifying visual attention in mobile HCI?

   Mihai Bâce, Sander Staal, and Andreas Bulling

   IEEE Pervasive Computing, Oct 2020

   DOI
2. Combining Gaze Estimation and Optical Flow for Pursuits Interaction

   Mihai Bâce, Vincent Becker, Chenyang Wang, and Andreas Bulling

   In Proc. ACM International Symposium on Eye Tracking Research and Applications (ETRA), Oct 2020

   DOI
3. Quantification of Users’ Visual Attention During Everyday Mobile Device Interactions

   Mihai Bâce, Sander Staal, and Andreas Bulling

   In Proc. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), Oct 2020

   DOI Video

2019

1. Accurate and Robust Eye Contact Detection During Everyday Mobile Device Interactions

   Mihai Bâce, Sander Staal, and Andreas Bulling

   ArXiv, Oct 2019

2018

1. Wearable Eye Tracker Calibration at Your Fingertips

   Mihai Bâce, Sander Staal, and Gábor Sörös

   In ACM Symposium on Eye Tracking Research & Applications, Oct 2018

   DOI

2017

1. Collocated Multi-user Gestural Interactions with Unmodified Wearable Devices

   Mihai Bâce, Sander Staal, Gábor Sörös, and Giorgio Corbellini

   Augmented Human Research, Oct 2017
2. Augmenting Human Interaction Capabilities with Proximity, Natural Gestures, and Eye Gaze

   Mihai Bâce

   In Proceedings of the 19th ACM International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2017), Oct 2017

   DOI
3. Facilitating Object Detection and Recognition through Eye Gaze

   Mihai Bâce, Philippe Schlattner, Vincent Becker, and Gábor Sörös

   In Proceedings of the Workshop on Object Recognition for Input and Mobile Interaction at the 19th ACM International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2017), Oct 2017

   DOI
4. HandshakAR: Wearable Augmented Reality System for Effortless Information Sharing

   Mihai Bâce, Gábor Sörös, Sander Staal, and Giorgio Corbellini

   In Proceedings of the Augmented Human 2017 Conference (AH 2017), Oct 2017

   DOI

2016

1. ubiGaze: Ubiquitous Augmented Reality Messaging Using Gaze Gestures

   Mihai Bâce, Teemu Leppänen, Argenis Ramirez Gomez, and David Gil Gomez

   In SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications, Oct 2016

   DOI

2015

1. Lightweight Indoor Localization System

   Mihai Bâce and Yvonne-Anne Pignolet

   In Proceedings of the 8th IFIP Wireless and Mobile Networking Conference (WMNC 2015). Munich, Germany., Oct 2015

   DOI

2011

1. Lane identification and ego-vehicle accurate global positioning in intersections

   Voichita Popescu, Mihai Bâce, and Sergiu Nedevschi

   In Intelligent Vehicles Symposium (IV 2011). Baden-Baden, Germany., Oct 2011

   DOI
2. Probabilistic Approach for Automated Reasoning for Lane Identification in Intelligent Vehicles

   Voichita Popescu, Mihai Bâce, and Sergiu Nedevschi

   In International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC 2011). Timisoara, Romania., Oct 2011

   DOI