PhD research: Multimodal story generation using generative AI for contextualised mathematics education

Challenges of traditional teaching methods

Mathematics is an important subject within the school curriculum, but students often find it challenging and tedious. One key reason is that traditional teaching methods are textbook-based and teacher-centred. In such teaching methods, mathematical concepts are typically taught in an abstract and formulaic way, which is different from how students might experience these concepts in the real world.

These challenges might be more prominent for children aged 7 to 11. According to Piaget's Theory of Cognitive Development, children in this age group are at the concrete operations stage, where they develop logical and concrete reasoning but have not acquired the ability for abstract and hypothetical thinking. One study noted that teaching mathematics to children at this stage requires helping them make connections between concepts and practical activities.

Advantages of context-based learning and educational stories

Compared to the traditional teaching approach, context-based learning is a student-centred approach that incorporates real-life and fictitious examples, emphasising practical experience over theory. Accordingly, educational stories could serve as a tool within this approach, which aims to connect academic theory with practice through engaging narratives and visual content. For children aged from 7-11, research indicates that they engage more effectively with content presented in story form, as narratives provide an intuitive way to process and retain information.

Example of story generated by our LLM-based framework.

Generative AI for educational stories

When using educational stories for context-based learning, the stories need to be interesting to children. However, creating tailored (e.g. customised and personalised) educational stories for each child and each mathematical problem is challenging for educators in terms of the time and resources required. Recent advancements in generative artificial intelligence (GenAI), including large language models (LLMs) for text generation and diffusion models for image generation, enable the automated creation of personalised narratives and visuals. Therefore, GenAI is increasingly used to enhance student engagement and comprehension through personalised learning materials.

Designing the story generation system

Overview of GenAI-driven context-based learning system.

This research aims to develop a context-based learning system that leverages generative AI models to automatically create personalised, multimodal mathematics stories from user-provided mathematical problems. These stories are designed to help children better understand mathematical concepts and support them in problem-solving. 

The system design is informed by REACT learning strategies from context-based learning: 

• Relating: Link new knowledge to students’ prior experiences.
• Experiencing: Encourage active exploration and discovery.
• Applying: Use learned concepts in practical situations.
• Cooperating: Support collaborative learning and peer discussion.
• Transferring: Apply knowledge to new contexts and problems.

The system comprises two core components: a story generation module and an interactive learning module. Both modules exclusively employ open source generative AI models, deployed on consumer-grade computing devices through portable machine learning formats. This design eliminates reliance on external resources while ensuring both privacy and accessibility.

Current stage: story text-to-text generation

Ensuring the reliability of generated story text is critical when embedding mathematical content into educational stories. Since children rely on these stories not only for engagement but also for conceptual understanding, any factual or logical inconsistency might lead to confusion or misconceptions. Previous studies have shown that using an LLM-based multi-agent framework, which breaks down the writing task into subtasks, can result in more accurate, structured and coherent narratives. Building on this approach, we constructed a framework comprising three dedicated LLM-based agents responsible for mathematical problem solving, story planning and story writing. 

For evaluating our framework, we generated mathematics stories using LLaMA-3.1 in the Edinburgh International Data Facility's GPU service, both with and without the multi-agent framework, and assessed quality using LLM-as-Judge evaluation. The results show that the stories generated using the multi-agent framework are more accurate and relevant regarding the mathematical content, and also improve on the overall structure and logical flow. 

Currently, we are collaborating with researchers to conduct a user study with children, aiming to gain a deeper understanding of their perceptions of the generated stories in terms of story quality, interest and engagement.

Future work

Building on our current work, we will continue to explore multimodal (e.g. visual and audio) content generation for story-based and interactive learning to enhance learning experiences. We aim to provide personalised educational materials, particularly educational stories, to support children and teachers, and we hope our system can contribute to the development of other context-based learning tools and systems.

Get involved or find out more

If you are interested in participating in the upcoming user studies, exploring potential research collaborations, or reading examples of the mathematics stories generated by our system, please contact me: K.Zhang-61@sms.ed.ac.uk.

Paper presentation

I presented my paper “Multimodal Story Generation Using Generative AI for Contextualised Mathematics Education” at the 26th International Conference on Artificial Intelligence in Education (AIED 2025), Palermo, Italy in July 2025. Authors: Kejia Zhang, PhD student, EPCC; Dr Charaka Palansuriya, EPCC; Dr Aurora Constantin, School of Informatics.

Author

Kejia Zhang, PhD Student, EPCC, University of Edinburgh. 

Email: K.Zhang-61@sms.ed.ac.uk

PhD study at EPCC

For PhD opportunities at EPCC and an overview of our current students' research, see our PhD study webpages.