StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesis

To leverage LLMs for visual synthesis, traditional methods convert rasterimage information into discrete grid tokens through specialized visual modules,while disrupting the model's ability to capture the true semanticrepresentation of visual scenes. This paper posits that an alternativerepresentation of images, vector graphics, can effectively surmount thislimitation by enabling a more natural and semantically coherent segmentation ofthe image information. Thus, we introduce StrokeNUWA, a pioneering workexploring a better visual representation ”stroke tokens” on vector graphics,which is inherently visual semantics rich, naturally compatible with LLMs, andhighly compressed. Equipped with stroke tokens, StrokeNUWA can significantlysurpass traditional LLM-based and optimization-based methods across variousmetrics in the vector graphic generation task. Besides, StrokeNUWA achieves upto a 94x speedup in inference over the speed of prior methods with anexceptional SVG code compression ratio of 6.9