Summary
A mock-up showing a possible analysis dialogue where visualisations are generated through natural language interaction and the responses are given through a mix of visualisation and language. This vision requires us to build a
thorough understanding of how people express their understanding and build queries when encountered with visualisations
The unprecedented increase in the amount, variety and value of data has been significantly transforming the way that scientific research is carried out and businesses operate. As data sources become increasingly diverse and complex, analysis approaches where the human and the computer operate in collaboration have proven to be an effective approach to derive actionable observations. This is achieved through an iterative human-computer dialogue where the knowledge and the creativity of the human meets the power of computation. In such human-in-the-loop data analysis approaches, interactive visualisation methods are core facilitators of this dialogue. However, these methods still rely on conventional, not often intuitive interaction mechanisms that can introduce unnecessary complexities into the process. There is an urgent need to rethink the ways how analysts interact with visualisations in data-intensive analysis situations. The recent advances in natural language based interaction methodologies offer promising avenues to address that.
This project aims to develop a fundamental understanding of how analysts can use natural language elements to perform visualisation empowered data analysis and use that understanding to develop a framework where natural language and visualisation based interactions operate in harmony. The project then aims to demonstrate how such a multi-modal interaction scheme can radically transform the analysts' experience with the goal of achieving significant improvements in the value and the volume of actionable observations generated.