Bridging Generative Models and Visual Communication

Abstract: From rough sketches that spark ideas to polished designs that explain complex concepts, visual communication is central to how humans think, create, and share knowledge. Yet despite major advances in generative AI, we are still far from models that can reason and communicate through visual forms. I will present my work on bridging generative models and visual communication, focusing on three complementary domains: (1) algorithms for generating and understanding sketches, (2) systems that support exploratory visual creation beyond one-shot generation, and (3) methods for producing editable, parametric images for design applications. These domains pose unique challenges; they are inherently data-scarce and rely on representations that go beyond pixel-based images commonly used in standard models. I will show how the rich priors of vision-language models can be leveraged to address these challenges through novel optimization objectives and regularization techniques that connect their learned features with the specialized representations required for visual communication. Looking ahead, this research lays the foundation for general-purpose visual communication technologies: intelligent systems that collaborate with humans in visual domains, enhancing how we design, learn, and exchange knowledge.