Absolutely fascinating work by @SakanaAILabs reproducing @kenneth0stanley Picbreeder in a non-interactive, VLM-agentic way.
I've had years to reflect on Kenneth Stanley's ideas as originally communicated in the book Why Greatness Cannot Be Planned.
The most load-bearing term is "serendipity", and what that actually means.
> Weâre inspired by the idea that serendipity requires chance
As we laid out in our creativity article, I think chance is actually the wrong way to think about serendipity. It's mostly about respecting constraints, and the chance, if there is any, happens in the degrees of freedom not covered by the constraints. I think open-endedness can be thought of in a privileged, symbolic/non-statistical way.
The more recent work with Kenneth and @akarshkumar0101 elucidated to me that it's about three main things: First of all, the lack of any externalised agency ("following your own gradient of interest") or only non-complex internal agency or intelligence (as explained in his original book, framed there as "non-optimization"), secondly -- "deep understanding" (respecting constraints upstream in the phylogeny), or, in their terms "path dependence". And thirdly: "evolvability". It's also taken as a given that if the constraints are sparse/factorised/world-aligned, the more the better. Actually, the best possible sign of success of any representation learning architecture is sparsity, factorization, world-alignment and/or canalisation.
I was able to breed the anthropomorphic images below by manually navigating an extremely deceptive search space over many, many iterations and starting with something that didn't look anything like a face. The reason Picbreeder works through interactive human supervision is that we understand the world at an abstract level, and we can easily spot abstract motifs in the generated images. As @fchollet said, the ability to decompose perceptual information into its abstract constituent blocks is a core feature of human cognition. So Picbreeder was bootstrapping the iterative creation of a highly factorised modular architecture, by leveraging humans' ability to do just this.
It's interesting that the Sakana guys referred to serendipity as a "je ne sais quoi". It is paradoxical because it IS the state of understanding abstractly, perhaps without the conscious awareness of it. It's certainly true that when I was breeding the image below, I locked into a particular feature, and it became a convergent, "goal-orientated process". But the important thing is it was completely individual agency. I was following my own path, and it was extremely shallow agency. I wasn't trying to achieve a complex objective. I was merely trying to encourage the materialisation of the abstract feature I initially recognised.
When we look at the phylogeny created in this non-interactive way, we see that deep abstractions are not locked in and shared in deep lineages above.
I think the two main problems of automating this process with LLMs are: convergence in sampling, and the inability to understand perceptual information in an abstract and systematic way. Both of these problems are directly explained by the fractured entangled representation hypothesis. So if this was an experiment trying to find if we could produce a factorised phylogeny from a non-factorised phylogeny (LLMs), it seems to have proven otherwise at this scale. However, we shouldn't give up so easily, if there is, indeed, even the slightest gradient of factorization, this process could presumably be scaled and repeated. I'm also intrigued by some of the work from Goodfire, which seems to suggest that factorization does meaningfully increase with scale, even on SGD networks (@banburismus_ ).
Anyway, in my opinion, this is one of the most important research directions in the whole field of AI: leveraging open-ended methods and building networks with factorised representations. Looking forward to lots more exciting work from Sakana!
VLMã¯äººéã®ãããªåµé æ§ãæãŠããïŒ
ã±ãã¹ã»ã¹ã¿ã³ã¬ãŒææãã®ãç®æšãšããå¹»æ³ïŒWhy Greatness Cannot Be PlannedïŒãã¯ãæç¢ºãªç®æšãèšå®ããããšããããã£ãŠçã«å倧ãªçºèŠãé ãããŠããŸããšããé説ãè«ããæžç±ã§ãããã®è°è«ã®äžæ žã«ãã£ãã®ããPicBreederãã®å®éšã§ããã
PicBreeder ã§ã¯ããŠãŒã¶ãŒããé¢çœãããšæããç»åãéžã³ããããå°ããã€é²åãããŠãããŸããäºåã«æ±ºãããããŽãŒã«ã¯ãªãã人ã
ãããªããšãªãè¯ãããšæã£ããã®ãéžã³ç¶ããã ãã§ãé¡ãåç©ãä¹ãç©ãé è骚ãšãã£ãäºæããªã圢ããäœäžä»£ããããŠãå€ãã®äººã®æãçµãŠèªç¶ãšçŸããŸããã¹ã¿ã³ã¬ãŒææãã¯ãããããããªãŒãã³ãšã³ãããã€ãŸãç®æšããããããå®ããªãæ¢çŽ¢ããã人éã®åµé æ§ã®æ ¹å¹¹ã«ãããšèããã®ã§ãã
ã§ã¯ããã®ãªãŒãã³ãšã³ããªæ¢çŽ¢ããAIã¯åçŸã§ããã®ã§ããããã
MITã»NYUãšã®å
±åç ç©¶ãšããŠçºè¡šãããIn Search of the Ingredients of Open-Endedness: Replicating Picbreeder with Large Vision-Language Modelsãã§ã¯ãèŠèŠèšèªã¢ãã«ïŒVLMïŒãšãŒãžã§ã³ãã«ããåçŸã詊ã¿ãŸããããšãŒãžã§ã³ããã¡ã¯å
±æã¢ãŒã«ã€ããæ¢çŽ¢ããç»åãéžãã§é²åãããæ°ã«å
¥ã£ããã®ãå
¬éããä»ã®ãšãŒãžã§ã³ãã®äœåãè©äŸ¡ããŸããç®æšãšãªãç»åãã鲿©ãã®å®çŸ©ãäžããããŠããŸããã
ãã®çµæãAIã«ãããªãŒãã³ãšã³ããªçºèŠã®å¯èœæ§ãšéçã®äž¡æ¹ãæµ®ãã³äžãããŸãããVLMãšãŒãžã§ã³ãã¯ç¹å®ã®èŠãç®ãæå³ã«åŒãå¯ãããããããæ¢åã®ã¢ã€ãã¢ãæšãŠãŠäºæãã¬äœããæ¢ãããããæå
ã«ãããã®ãæŽç·Žãããããšã«çãŸããã¡ã§ãããäžæ¹ã§ã倿§ãªäººæ Œãæã€ãšãŒãžã§ã³ãéå£ãå°å
¥ãããšæ¢çŽ¢ã¯å€§ããæ¹åãããçæãããã¢ãŒã«ã€ãã®æå³çãªå¹
åºãã¯ã人éãäœã£ãã¢ãŒã«ã€ãã«è¿«ãæ°Žæºã«ãŸã§éããŸããã
ããããVLMã§ã¯å±ããªãã£ãç¹ããããŸããã人éã¯ãå¶ç¶ã®ç£ç©ãæç¶çãªåµé ãžãšã€ãªããããšã«é·ããŠããŸãããããã®ãèŠã€ãããšãããã®äŸ¡å€ãæãåããããã远ããããããšã§ããã倧ããªæŠå¿µçé£èºãéæã§ããŸããAIãšãŒãžã§ã³ããè峿·±ããã¿ãŒã³ã«æ°ã¥ãããšã¯ã§ããŠãããã®ãã¿ãŒã³ã«åãããŠããŸãåŸåãã¿ãããŸããã
ãªã人éã¯ãæ¬ç ç©¶ã®VLMã«ã¯ã§ããªãã£ããªãŒãã³ãšã³ããªæ¢çŽ¢ãé²ããããã®ããçŸåšã®AIã·ã¹ãã ã«äœãæ¬ ããŠããã®ããç§ãã¡ã¯ãŸã ååã«çè§£ã§ããŠããŸãããããã«ã¯ãåœç€Ÿãæ¢æ±ããAIé§ååç§åŠç ç©¶ã«ãéãã倧ããªåããæ®ã£ãŠããŸããSakana AIã¯ä»åŸãããªãŒãã³ãšã³ããªç¥æ§ã®æ¢æ±ãæ·±ããŠãããŸãã
ããã°ïŒpub.sakana.ai/picbreeder-vlm
è«æïŒarxiv.org/abs/2605.23908 ð
The AI Picbreeder Experiment: Can AI agents be creative when nobody tells them what to create?
Blog: pub.sakana.ai/picbreeder-vlm
In our new #gecco2026 paper, "In Search of the Ingredients of Open-Endedness: Replicating Picbreeder with Large Vision-Language Models", in collaboration with MIT and NYU, we revisit Picbreeder, a lost website where people collaboratively evolved images without any predefined objective. Users simply selected images they found interesting, allowing unexpected forms such as faces, animals, vehicles, and skulls to emerge gradually across many generations and many different people.
We recreated this process using vision-language model agents. The agents explore a shared archive, choose images to branch from, evolve new candidates, publish their favorites, and evaluate the creations of other agents. There is no target image and no explicit definition of what counts as progress.
The results reveal both the promise and current limitations of AI-driven open-ended discovery.
Compared with humans, VLM agents tend to keep circling back to the same kinds of images and concepts. They repeatedly select similar parents, make smaller conceptual leaps, and often refine an existing idea rather than abandoning it in search of something genuinely unexpected.
However, introducing a diverse population of agent personalities substantially improves exploration. In some runs, diverse agent populations approached or matched the human archive on measures of semantic diversity and produced more balanced evolutionary trees.
We also find intriguing evidence that open-ended evolution can produce more robust representations. A skull evolved by the agents changes smoothly when its underlying neural representation is perturbed, less fractured than a skull directly optimized with gradient descent, although still less cleanly disentangled than one evolved collectively by humans.
But perhaps the most interesting result is the gap that remains.
Humans appear better at turning fortunate accidents into sustained creative discoveries: recognizing when something unexpected is worth pursuing, refining it, and then making a larger conceptual leap. The AI agents often notice interesting patterns too, but are more likely to become trapped in them.
We still do not fully understand what enables humans to navigate open-ended search in this way, or what ingredient(s) current AI systems are missing. For now, the results suggest that there remains something important about human creativity that AI agents have not yet learned to reproduce.
This paper will be presented at #gecco2026 and is nominated for a best paper award! Please check out the interactive blog and technical paper for more details!
Read our full paper: arxiv.org/abs/2605.23908 ð