Enhanced Ontological Visual Synthesis of Classical Chinese Poetry

Document-Enhanced Ontological Visual Synthesis of Classical Chinese Poetry: Open-Source Multi-Sample Bayesian Approach

We introduce a framework for visualizing Classical Chinese poetry using AI, addressing limitations in existing approaches. The system uses three key innovations: (1) an ontology-driven pipeline connecting poetic concepts to visual elements, (2) Bayesian-optimized parameters for Stable Diffusion XL, and (3) dynamic visualization with fluid transitions between poetic segments. Built on open-source models, our framework shows improved visual quality and consistency in representing both abstract and concrete poetic elements compared to current methods.

Project Summary:

Classically rich in imagery and multiple meanings, Chinese poetry has presented many unique challenges to modern AI visualization systems. While recent text-to-image generation developments have been remarkably capable, classical Chinese poetry

remains challenging to accurately visualize because of the sophisticated use of metaphor, cultural context, and abstract concepts that go beyond literal interpretation.

Current methods for visualization apply a direct translation approach that is unable to capture detailed relationships between the abstract poetic concepts and their concrete visual representations (happens in DALLE3, MidJourney, Flux). The inherent stochasticity of the modern diffusion models further exacerbates these issues when dealing with culturally-specific artistic elements. We utilize Stable Diffusion XL as our foundational model due to its

open-source nature enabling deep parameter optimization, built-in refiner for quality enhancement, and accessibility for reproducible research.

Our novel end-to-end framework combines ontological knowledge integration with optimized image generation and dynamic video synthesis. Technical contributions include a custom ontology-driven pipeline that bridges abstract concepts with visual elements (some similarity with HyDE pipeline but different), a Bayesian-optimized guidance scale parameter tuning system for SDXL (optimal w = 9.8) to sharpen and strength representations, and a dynamic visualization approach using sliding-window interpolation between poetic half-stanzas. With extensive experiments, we demonstrate significant improvements in both quantitative metrics and qualitative assessments, establishing a foundation for culturally aware AI systems capable of handling complex literary works.

It is currently a project being targeted for a paper under review at a high venue.