The intersection of design, artificial intelligence, and data-driven storytelling is giving rise to a new era of human-machine collaboration. Researchers are exploring the potential of speculative design, multimodal large language models (MLLMs), and narrative visualization to transform the way we create and interact with information.
Speculative design, a discipline that uses "what if?" scenarios to explore possible futures, is being reframed through an information-theoretic lens. According to a recent study, speculative design can be seen as a resource-bounded knowledge generation process that uses prototyping to strategically embrace surprise (Source 1). However, not all surprises are equally informative, and designers need to be able to distinguish between genuine insight and aesthetic shock.
The integration of MLLMs into early-stage design tools is also changing the way designers collaborate with AI. A user study with 12 participants analyzed sketch-based design interactions with an MLLM-powered system and found that designers rarely rely on a single mode of interaction; instead, human-led and AI-led roles are frequently interwoven and shift across ideation instances (Source 2). This dynamic collaboration has implications for the development of interactive systems that can support designers in their creative process.
In the field of education, codesigning with teachers is leading to the development of innovative tools that support assessment authoring. Ripplet, an LLM-assisted assessment authoring system, was codesigned with 13 teachers over a seven-month period and has been shown to enable teachers to create formative assessments they would not have otherwise made (Source 3).
Narrative medical visualization is another area where researchers are applying data-driven storytelling techniques to communicate complex scientific results to a general audience. By merging exploratory and explanatory visualization, researchers can create interactive and engaging visualizations that help non-experts understand medical research results (Source 4).
The use of conceptual metaphors in scientific storytelling is also being explored, with researchers applying Conceptual Metaphor Theory (CMT) to the visualization domain. A classification of visual conceptual mappings within scientific representations has been developed, providing a taxonomy and grammar for understanding and describing metaphor use in visualization (Source 5).
These developments have significant implications for the way we design, interact with, and understand complex information. As AI becomes increasingly integrated into various domains, the need for effective human-machine collaboration and data-driven storytelling will only continue to grow. By leveraging speculative design, MLLMs, and narrative visualization, researchers and designers can create innovative tools and systems that support creativity, collaboration, and comprehension.
References:
- "Speculating for Epiplexity: How to Learn the Most from Speculative Design?" arXiv:2602.22132v1
- "A Taxonomy of Human--MLLM Interaction in Early-Stage Sketch-Based Design Ideation" arXiv:2602.22171v2
- "Codesigning Ripplet: an LLM-Assisted Assessment Authoring System Grounded in a Conceptual Model of Teachers' Workflows" arXiv:2602.22186v1
- "Towards Narrative Medical Visualization" arXiv:2108.05462v1
- "The Language of Infographics: Toward Understanding Conceptual Metaphor Use in Scientific Storytelling" arXiv:2407.13416v1
The intersection of design, artificial intelligence, and data-driven storytelling is giving rise to a new era of human-machine collaboration. Researchers are exploring the potential of speculative design, multimodal large language models (MLLMs), and narrative visualization to transform the way we create and interact with information.
Speculative design, a discipline that uses "what if?" scenarios to explore possible futures, is being reframed through an information-theoretic lens. According to a recent study, speculative design can be seen as a resource-bounded knowledge generation process that uses prototyping to strategically embrace surprise (Source 1). However, not all surprises are equally informative, and designers need to be able to distinguish between genuine insight and aesthetic shock.
The integration of MLLMs into early-stage design tools is also changing the way designers collaborate with AI. A user study with 12 participants analyzed sketch-based design interactions with an MLLM-powered system and found that designers rarely rely on a single mode of interaction; instead, human-led and AI-led roles are frequently interwoven and shift across ideation instances (Source 2). This dynamic collaboration has implications for the development of interactive systems that can support designers in their creative process.
In the field of education, codesigning with teachers is leading to the development of innovative tools that support assessment authoring. Ripplet, an LLM-assisted assessment authoring system, was codesigned with 13 teachers over a seven-month period and has been shown to enable teachers to create formative assessments they would not have otherwise made (Source 3).
Narrative medical visualization is another area where researchers are applying data-driven storytelling techniques to communicate complex scientific results to a general audience. By merging exploratory and explanatory visualization, researchers can create interactive and engaging visualizations that help non-experts understand medical research results (Source 4).
The use of conceptual metaphors in scientific storytelling is also being explored, with researchers applying Conceptual Metaphor Theory (CMT) to the visualization domain. A classification of visual conceptual mappings within scientific representations has been developed, providing a taxonomy and grammar for understanding and describing metaphor use in visualization (Source 5).
These developments have significant implications for the way we design, interact with, and understand complex information. As AI becomes increasingly integrated into various domains, the need for effective human-machine collaboration and data-driven storytelling will only continue to grow. By leveraging speculative design, MLLMs, and narrative visualization, researchers and designers can create innovative tools and systems that support creativity, collaboration, and comprehension.
References:
- "Speculating for Epiplexity: How to Learn the Most from Speculative Design?" arXiv:2602.22132v1
- "A Taxonomy of Human--MLLM Interaction in Early-Stage Sketch-Based Design Ideation" arXiv:2602.22171v2
- "Codesigning Ripplet: an LLM-Assisted Assessment Authoring System Grounded in a Conceptual Model of Teachers' Workflows" arXiv:2602.22186v1
- "Towards Narrative Medical Visualization" arXiv:2108.05462v1
- "The Language of Infographics: Toward Understanding Conceptual Metaphor Use in Scientific Storytelling" arXiv:2407.13416v1