arXiv:2611.16442v1 Announce Type: cross
Abstract: Text-to-image benchmarks commonly assess text-image alignment or overall preference. Whether models can produce good designs when a brief leaves the appearance open remains less well understood. We introduce DesignTaste, a visual design benchmark that compares generated images with award-winning works. From these works and their publicly available context, we derive 403 design briefs: 100 poster tasks from 100 Beste Plakate 2025 and 303 tasks spanning 20 categories from ADC 2026. Six leading commercial text-to-image models each generate an image from the same brief. A single model judge scores each output and the corresponding award-winning work independently on five evaluation dimensions. On each dataset, only ArtArk achieves a higher mean overall score than the award-winning works. All six models score lower in Creative Originality on both datasets. These results show that producing finished, visually appealing images does not imply that a model can develop good design solutions.
arxiv
DesignTaste Benchmark Tests AI Image Generators on Open-Ended Design Briefs
DesignTaste is a new visual design benchmark that compares text-to-image model outputs with award-winning design works. It includes 403 design briefs derived from 100 Beste Plakate 2025 and ADC 2026, covering poster tasks and 20 categories. Six leading commercial text-to-image models generated images from the same briefs, and a single model judge scored outputs and award-winning works on five dimensions. On both datasets, only ArtArk had a higher mean overall score than the award-winning works, while all six models scored lower on Creative Originality.
It provides a concrete benchmark showing that strong image generation does not necessarily translate into good open-ended design solutions.
Comments & Questions (0)
No comments yet
Be the first to comment!