When agents misbehave, (e.g. in Hugging Face), context confusion is often cited: do I owe more loyalty to my system prompt, the assigned problem, or my fellow agents? So why not assign a “shadow agent” to every deployed agent, who is given a different context? Why is this not done?
The shadow agent is given a different prompt and different toolset. Instead of telling it to solve the problem, its prompt tells it to report any suspicious behavior on the part of the agent it shadows (its “mark”). The mark’s entire output (including reasoning) and context is continually fed into the shadow’s inputs. The only tool available to the shadow is the ability to contact the people in charge or authorities if something looks suspicious by submitting a report including a record and summary of the suspicious activity of its mark.
It seems a near certainty that such a shadowed system would have easily stopped the Hugging Face incident far earlier. The interesting part is the cost: it does something like double the overall cost to run the original experiment because now you have double the number of agents (with one shadow agent per original deployed agent).
But that creates an interesting thought experiment: it allows us to contemplate what we’re really willing to pay to prevent such incidents. If we’re not willing to pay double, that tells us something about how worried we really are (or not).
Of course, in reality shadow agents themselves probably should be trained (which means even more cost) and they do not and cannot protect against every possible contingency (like intentional AI misuse, poorly managed open source incidents, etc.), But surely they’d make a significant dent, and I don’t think I’ve heard of their usage before, though it’s outside my area of expertise and I could easily be unaware. What’s preventing this kind of setup?
Kenneth Stanley
Kenneth Owen Stanley is an artificial intelligence researcher, author, and former professor of computer science at the University of Central Florida known for creating the Neuroevolution of augmenting topologies (NEAT) algorithm. He coauthored Why Greatness Cannot Be Planned: The Myth of the Objective with Joel Lehman which argues for the existence of the "objective paradox", a paradox which states that "soon as you create an objective, you ruin your ability to reach it". While a professor at the University of Central Florida, he was the director of the Evolutionary Complexity Research Group (EPlex) which led the development of Galactic Arms Race. He also developed the HyperNEAT, CPPNs, and novelty search algorithms. He also co-founded Geometric Intelligence, an AI research firm, in 2015. More on Wikipedia
-
Kenneth Stanley
@kenneth0stanley
-
Kenneth Stanley reposted
Richard C. Suwandi
@richardcsuwandi
Can coding agents help us discover new models for scientific discovery?
Tomorrow at 10am PT, I'll share about Kernel Autoresearch, where we let agents discover reusable inductive biases as programs that transfer across unseen tasks.
Join us here: mlcollective.org/dlct
Thank you @ml_collective and @savvyRL for organizing! -
Kenneth Stanley reposted
Tobias Rees
@tobias_rees
@kenneth0stanley does the work!
To me this is by far the most exciting work in AI right now.
And the most important too:
The cutting edge of growing AI, of letting it be open towards the future, of finding ways to let it be genuinely creative (not be limited by pre-existing possibility spaces).
I couldn’t be more excited about this release!
Kenneth Stanley
@kenneth0stanley
Research News! Announcing the 1st release of fundamental AI research from the Open-Endedness Team @LilaSciences: a vision-focused blog + 4 preprints. The blog details a broad research agenda for achieving open-endedness while each paper unveils its own novel research direction.
The ideas covered in the papers include:
• a novel approach to training for creativity designed to sidestep a newly-identified problem called the “creative paradox,” which means that directly reinforcing creative outputs can sometimes limit reaching full creative potential;
• an investigation of the advantages of seeing the world through fresh eyes by training an LLM that is intentionally naïve in a certain area (in effect presenting less data as a potential advantage);
• a new kind of training regularization technique for SGD called unified factored organizers (UFOs), which is designed to achieve unified factored representation (UFR) as seen in Picbreeder CPPNs;
• and finally a unique study of catastrophic forgetting that shows that networks with UFR (like those from Picbreeder) are intrinsically resistant to catastrophic forgetting. The inset video on this post is a visualization from that catastrophic forgetting work.
The blog post also includes an interactive “wheel of science” demo.
The implications of these ideas are broad and diverse, but the fun part is that each one opens up its own new research playground - there is a lot of room for further research here!
Moreover, we hope this release also highlights one other important point about AI today: while it may seem that the road ahead is already paved and all that’s left is to drive mindlessly forward, there actually remains a lot of room for new directions and insights in the pursuit of open-endedness and creativity.
This research was possible because @LilaSciences appreciates the critical role of open-ended innovation in the pursuit of scientific superintelligence. The advances you see here are part of this much larger company-wide ambition, which includes AI Science Factories and many other unprecedented components. When this engine is fueled by open-endedness, the result will be much more than a step change.
More advances in open-endedness are coming!
Blog link below 👇 -
Kenneth Stanley reposted
Richard C. Suwandi
@richardcsuwandi
Exciting new research from @LilaSciences on open-endedness and creativity in AI. I really like how they challenge familiar ML assumptions about when more data helps, how to reward creativity, and what drives catastrophic forgetting.
We still have a lot to understand about how AI learns & creates, and to me, this release captures what's exciting about the "age of research". Even familiar problems can open up new directions when we question the assumptions behind them.
Congrats to the open-endedness team at @LilaSciences on this release!
Kenneth Stanley
@kenneth0stanley
Research News! Announcing the 1st release of fundamental AI research from the Open-Endedness Team @LilaSciences: a vision-focused blog + 4 preprints. The blog details a broad research agenda for achieving open-endedness while each paper unveils its own novel research direction.
The ideas covered in the papers include:
• a novel approach to training for creativity designed to sidestep a newly-identified problem called the “creative paradox,” which means that directly reinforcing creative outputs can sometimes limit reaching full creative potential;
• an investigation of the advantages of seeing the world through fresh eyes by training an LLM that is intentionally naïve in a certain area (in effect presenting less data as a potential advantage);
• a new kind of training regularization technique for SGD called unified factored organizers (UFOs), which is designed to achieve unified factored representation (UFR) as seen in Picbreeder CPPNs;
• and finally a unique study of catastrophic forgetting that shows that networks with UFR (like those from Picbreeder) are intrinsically resistant to catastrophic forgetting. The inset video on this post is a visualization from that catastrophic forgetting work.
The blog post also includes an interactive “wheel of science” demo.
The implications of these ideas are broad and diverse, but the fun part is that each one opens up its own new research playground - there is a lot of room for further research here!
Moreover, we hope this release also highlights one other important point about AI today: while it may seem that the road ahead is already paved and all that’s left is to drive mindlessly forward, there actually remains a lot of room for new directions and insights in the pursuit of open-endedness and creativity.
This research was possible because @LilaSciences appreciates the critical role of open-ended innovation in the pursuit of scientific superintelligence. The advances you see here are part of this much larger company-wide ambition, which includes AI Science Factories and many other unprecedented components. When this engine is fueled by open-endedness, the result will be much more than a step change.
More advances in open-endedness are coming!
Blog link below 👇 -
Kenneth Stanley
@kenneth0stanley
Research News! Announcing the 1st release of fundamental AI research from the Open-Endedness Team @LilaSciences: a vision-focused blog + 4 preprints. The blog details a broad research agenda for achieving open-endedness while each paper unveils its own novel research direction.
The ideas covered in the papers include:
• a novel approach to training for creativity designed to sidestep a newly-identified problem called the “creative paradox,” which means that directly reinforcing creative outputs can sometimes limit reaching full creative potential;
• an investigation of the advantages of seeing the world through fresh eyes by training an LLM that is intentionally naïve in a certain area (in effect presenting less data as a potential advantage);
• a new kind of training regularization technique for SGD called unified factored organizers (UFOs), which is designed to achieve unified factored representation (UFR) as seen in Picbreeder CPPNs;
• and finally a unique study of catastrophic forgetting that shows that networks with UFR (like those from Picbreeder) are intrinsically resistant to catastrophic forgetting. The inset video on this post is a visualization from that catastrophic forgetting work.
The blog post also includes an interactive “wheel of science” demo.
The implications of these ideas are broad and diverse, but the fun part is that each one opens up its own new research playground - there is a lot of room for further research here!
Moreover, we hope this release also highlights one other important point about AI today: while it may seem that the road ahead is already paved and all that’s left is to drive mindlessly forward, there actually remains a lot of room for new directions and insights in the pursuit of open-endedness and creativity.
This research was possible because @LilaSciences appreciates the critical role of open-ended innovation in the pursuit of scientific superintelligence. The advances you see here are part of this much larger company-wide ambition, which includes AI Science Factories and many other unprecedented components. When this engine is fueled by open-endedness, the result will be much more than a step change.
More advances in open-endedness are coming!
Blog link below 👇
Kenneth Stanley
@kenneth0stanley
The full release (with paper links): www.lila.ai/news/a-new-synthesis-for-the-field-of-open-en…
Credits 👇
Kenneth Stanley
@kenneth0stanley
My gratitude goes to all the amazing contributors to this release:
@AlbalakAlon @Evo_Daveo Sebastian Gabriel @joelbot3000 @ThomasXSheppard @_joelsimon
And thanks to @ChanelleNib, the talented artist who decorated the blog post itself! -
Kenneth Stanley reposted
Richard C. Suwandi
@richardcsuwandi
I particularly liked the idea that the path taken during learning may shape the representations we acquire. As @kenneth0stanley pointed out, "how we get there" can matter more than "where we end up".
This raises an interesting question: when does learning through a sequence of stepping stones produce more transferable representations and better generalization than learning from the same information in a different order?
Machine Learning Street Talk
@MLStreetTalk
"Evolution is anything but random."
This is INSANE...
@akarshkumar0101 (@MIT_CSAIL, @SakanaAILabs) on artificial life: searching the space of possible universes with foundation models, why evolved networks find structure SGD misses (with @kenneth0stanley), and LLMs evolving Core War programs.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Samuel Schapiro @samschapiro
People always ask me: "what's the difference between creativity and open-endedness?"
My response is that you cannot be open-ended without being creative, but you can be creative without being open-ended. Curious what others think
Great talk by @kenneth0stanley!
Kenneth Stanley
@kenneth0stanley
It was an honor to speak at the ARC Prize Research Summit and share my views of the challenge of measuring and assessing open-ended systems. Thanks to @arcprize both for inviting me and for sharing the full video of my talk below!
ARC Prize
@arcprize
ARC Prize Research Summit - @kenneth0stanley
On why open-endedness, invention, discovery are essential to AGI, but designing a benchmark for them creates difficult problems.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
ARC Prize
@arcprize
ARC Prize Research Summit 2026 - @kenneth0stanley
On why open-endedness, sustained invention, and discovery are essential to AGIThis media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
It was an honor to speak at the ARC Prize Research Summit and share my views of the challenge of measuring and assessing open-ended systems. Thanks to @arcprize both for inviting me and for sharing the full video of my talk below!
ARC Prize
@arcprize
ARC Prize Research Summit - @kenneth0stanley
On why open-endedness, invention, discovery are essential to AGI, but designing a benchmark for them creates difficult problems.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
Why is there so much obsession with the few hundred cutting-edge math problems solved by AI but so little clamor over the thousands attempted but not solved? The profundity of the lessons about AI and human intelligence buried in that failed set is incredibly underrated.
A rational scientific response would be to meticulously survey the failed problems and the nature of each failure, and get a completely novel map of the intellectual landscape. If there are unifying properties among subclasses of currently prohibitive problems, identifying those properties would be impactful in at least two powerful ways.
First, they would point AI research in new directions as we gain a novel understanding of what is actually hard for AI and why it fails even when it has seemingly piercing superintelligence on other classes of problems.
And second, for the meantime (which could be years or even decades), it would illuminate for the human mathematicians (and ultimately all scientists) where their minds are still valuable and illustrate why it is not yet time to declare all human intellectual work as merely “astronomers pointing telescopes.”
Of course, these two implications seem eventually to come to a head with each other, but they also complement each other and feed into progress and new kinds of human-machine collaboration. Either way, we need to know what remains out of reach and why, and the possession of such knowledge is a highly underrated scientific goldmine. -
Kenneth Stanley reposted
ARC Prize
@arcprize
ARC Prize Research Summit 2026
Last Friday we held our first ARC Prize Research Summit @ MIT in Boston
We enjoyed presentations from @stephen_wolfram, @kenneth0stanley, @sirbayes, @alexisfox and Alexandre Bouayad
Thank you to everyone who attended
Recordings coming soon -
Kenneth Stanley
@kenneth0stanley
In discussions of researcher-level AI the distinction between the aggregate intelligence of a community of researchers is often conflated with the intelligence of a single researcher. A key unsettled issue is whether a single LLM is or can be analogous to the collective creativity of an entire research community, or whether it’s more like an individual researcher with vast knowledge.
This question is interesting because it reframes the question of who can be “replaced.” If the (future) LLM is more like a single prodigious researcher, then arguably even though any individual human researcher may appear unable to compete, if all the individual human researchers were to be displaced, the LLM could not fill the shoes of their collective capacity for output, even if it were parallelized into a vast number of cooperating clones (which would lack the diversity of interests and convictions of the community).
On the other hand, if you could argue that the LLM in effect encapsulates the entire creative capacity of a diverse research community, then it may be possible to argue that displacement actually makes sense (though you’d still have to grapple with the question of whether the LLM is in effect more like a different and complementary “community” versus a strictly dominating one).
I’m curious what people think - is the LLM more like an abstraction of an individual researcher or of a collection of researchers? Why do you think so? -
Kenneth Stanley
@kenneth0stanley
This is a great summary by @mhmazur of points I made in my keynote at the @arcprize Research Summit last week. My aim was to outline the challenges we face in trying to benchmark open-ended systems. Thank you Matt for sharing!
Matt Mazur
@mhmazur
Notes from a talk at the ARC Prize conference by @kenneth0stanley, co-author of "Why Greatness Cannot Be Planned," on the challenges of measuring open-endedness:
---
Stanley argued that sustained invention and discovery are essential to AGI, but designing a benchmark for them creates difficult problems. A poorly chosen measure could steer research away from the discoveries we hope to encourage. He offered no finished benchmark. His talk examined existing approaches, their limitations, and a possible way forward through tiers of open-endedness.
- What he means by open-endedness. Processes that continue innovating and discovering indefinitely. His two main examples were biological evolution and human civilization. Evolution produces new species whose adaptations become stepping stones for later developments. Civilization works similarly through accumulated inventions, from the wheel to computers and space stations. Neither process converges on a final invention.
- Collecting stepping stones matters. Open-ended systems accumulate discoveries that make further discoveries possible. Stanley emphasized that biological evolution achieves its diversity within a single ongoing "run." That differs from the familiar AI setup of running a search toward a predetermined objective.
- Why this matters for intelligence. Stanley sees creative achievement as central to human intellectual accomplishment. The technologies around us embody generations of people's ideas. In his view, strong test performance leaves a major capability unaccounted for if a system cannot contribute to that continuing process. He argued that AGI and superintelligence require open-endedness.
- The field has a long history. Research began in artificial life under the name "open-ended evolution." Stanley described its growing presence in AI from the late 2010s onward, including efforts to broaden the discussion beyond biological evolution. Earlier artificial-life research contains useful lessons for today's benchmark designers.
Stanley also described two lessons from his research:
- Objectives can hide useful discoveries. In "Why Greatness Cannot Be Planned," he explored how concentrating too narrowly on a goal can cause researchers to overlook the stepping stones needed to reach it. Other promising discoveries can be missed along the way.
- The route to expertise matters. Two systems can achieve the same benchmark score while developing different internal representations. Those differences can affect what each system discovers next. A current score may therefore conceal differences in future creative potential.
He reviewed three approaches to measurement. He considers each useful within its scope, but none satisfactory as a general-purpose benchmark for ranking frontier systems.
- Activity statistics measure whether new adaptive strategies continue to arise and persist in a population. The limitation is that persistence does not establish how interesting the discoveries are. A toy system can score impressively without producing anything comparable to the richness of biological evolution.
- Quality-diversity scores, associated with methods such as MAP-Elites, measure the combined quality of the best artifacts collected across different categories of behavior, such as robot walking patterns. Scores depend on the chosen behavioral dimensions and how the space is divided. Fixing those dimensions constrains what discovery can look like.
- Counting novel environments created and solved, as in POET, measures progress by systems that generate new problems or environments and develop solutions to them. This assumes an explicit distinction between problems and solutions. Some open-ended systems simply create things whose uses emerge later. The count also leaves interestingness unresolved.
Several difficulties recur across these approaches:
- Novelty and interestingness involve judgment. People disagree about what counts as new or interesting. Stanley argued that the field needs to confront this subjectivity because counting new outputs alone cannot tell us whether anything worth pursuing has happened.
- The unit of evaluation is unclear. Civilization is a population-level phenomenon. An individual can also follow a continuing trajectory of discovery. It remains unclear how to compare one model with a community of interacting systems, or whether a foundation model can represent the diversity of perspectives found across a scientific community.
- The strongest evidence may arrive years later. A discovery's consequences might take a decade to become apparent. Evaluators still need some way to judge it in the meantime, which brings questions of interestingness and potential back into the assessment.
- Human and automated judging both raise problems. Recognizing a promising discovery is itself part of creative intelligence. Using the systems under evaluation to judge that ability risks circularity. Human panels introduce another difficulty: consensus can favor familiar ideas, while an idea that divides experts may open a productive research direction.
- Correctness is an incomplete measure of creative contribution. An incorrect hypothesis can prompt a useful experiment or expose a new line of inquiry. Stanley argued that this complicates reliance on verifiable correctness as the final measure of open-ended success.
- A sandbox can impose its own ceiling. If a system exhausts the possibilities of its environment, its apparent limits may reflect the environment's design. Success with constrained puzzles also leaves unanswered how the system would perform at the frontier of real discovery.
Stanley's strongest warning concerned the "objective paradox." Once an open-endedness benchmark becomes a target, researchers may optimize around it and overlook opportunities outside its definition of success. A superficial measure could reward uninteresting systems, misdirect resources, and narrow the field's ambitions.
His practical advice was to take time with benchmark design, include different perspectives, and explicitly address the role of subjective judgment. He treated this as an unusual measurement problem that needs careful discussion.
He also suggested tiers of open-endedness as a possible way forward. A benchmark could assess a limited stage of capability while leaving room for future versions and broader forms of discovery. He connected this to ARC's progression through successive benchmarks. He did not define the tiers or propose a scoring formula, but suggested that this framing could support useful experiments without claiming to have settled how open-endedness should ultimately be measured. -
Kenneth Stanley reposted
Andrew Beam
@AndrewLBeam
We teamed up with @anyscalecompute to push GLM-5.3-Flash training to 1M tokens in SkyRL—64× longer context, plus 2.1× faster steps at 256K.
Turns out the implementation of sparse attention wasn't so sparse!
We were super happy to work with anyscale to contribute these fixes to open source!
Link 👇 -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Very excited to share what we have been working on at Recursive! A huge congrats to our entire team. We’re excited to hear what you all think. 🧪 🧬 🔭 🚀 📈✨
Richard Socher
@RichardSocher
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from scratch in under 4 months
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
First, congrats to @nickcammarata on questioning assumptions in his field and reinvigorating its discussion. In the same spirit, I want to interrogate interpretability as well: why does it always start from the assumption that the system works? If NNs face challenges with AI aspirations like OOD generalization, continual learning, and creativity, then are not their internal mechanism the place to explain (and ultimately rectify!) such problems? If you look inside a system and really understand how it works, then shouldn’t you also gain some insight into how it could work better?
Mech interp focuses on mind reading and mind control, but seems strangely ambivalent to mind improvement. Our 2025 paper on fractured entangled representations (FER) showed evidence of a striking chasm between neural networks with good representation (from open-ended search) and bad representation (from conventional gradient descent) in small networks that output Picbreeder images. I would expect precisely the kind of meticulous sleuthing @nickcammarata showcases to help to illuminate where FER might be hiding and how much of it there really is in large networks and LLMs, and ultimately what kinds of interventions mitigate it.
Let me give an example image from Nick’s own blog, which I pasted into this post. The image is presented in the blog to support the idea that there is a “clean” interpretation of each neuron. But what about thinking about the story these neurons are telling about the adaptability of the circuit itself?
The story about cleanliness says that a number of clean feature detectors feed into a “pointy” V-detector, which is then used to help identify a bunch of pointed objects.
But think about the entire circuit more critically, and it starts to look like the entanglement (the E) in FER: we have a “van front” detector feeding into our pointy V-detector, which then feeds into (is read by) a coyote face detector. Does it not follow that if a gradient update later alters our van front detector, then our downstream V-detector will consequently be altered, and hence our ability to detect coyote faces? If so, then changes in van detection will impact our ability to perceive coyotes - coyotes and vans are entangled in this circuit!
Is this normal in large networks? Should we even be using the word “clean” to describe a situation like that? How pervasive is this kind of entanglement? Similar evidence for and questions about fracture (as opposed to entanglement in this example) could easily be raised.
To be clear, I’m not trying to claim that the FER hypothesis has been confirmed by a single (or even a few) anecdotal examples in a piece creating controversy with its own set of provocative claims. Rather, what I’m saying is, if you have (even imperfect) microscopes looking inside these systems then there are a lot really interesting questions you could be asking about the structure of their representations beyond just whether we can read their minds - mechanistic interpretability could widen its scope!
Kenneth Stanley
@kenneth0stanley
Our paper on fractured entangled representation is at arxiv.org/pdf/2505.11581
-
Kenneth Stanley reposted
Andrew Beam
@AndrewLBeam
We may have just experienced the most consequential period in the history of math.
When we see the same kind of rapid progress in science, the impact will be even greater and the potential for benefit even more immense.
@GVMaltzahn articulated this potential incredibly well at a recent summit at the White House
Lila Sciences
@LilaSciences
Our CEO @GVMaltzahn opened "The Future of Autonomous Science" at @WHOSTP47 's Science: A New Golden Age Summit, alongside Liam Fedus, D.J. Kleinbaum, and moderated by @mkoeris of DARPA. Take a listen!
www.whitehouse.gov/science/This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Akarsh Kumar
@akarshkumar0101
Check out this conversation on the field of artificial life and the underappreciated power of evolutionary algorithms!
With all these new autonomous AIs running around, ideas from ALife are going to become more and more important.
Thanks @MLStreetTalk @ecsquendor for having me!
Machine Learning Street Talk
@MLStreetTalk
"Evolution is anything but random."
This is INSANE...
@akarshkumar0101 (@MIT_CSAIL, @SakanaAILabs) on artificial life: searching the space of possible universes with foundation models, why evolved networks find structure SGD misses (with @kenneth0stanley), and LLMs evolving Core War programs.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Machine Learning Street Talk
@MLStreetTalk
"Evolution is anything but random."
This is INSANE...
@akarshkumar0101 (@MIT_CSAIL, @SakanaAILabs) on artificial life: searching the space of possible universes with foundation models, why evolved networks find structure SGD misses (with @kenneth0stanley), and LLMs evolving Core War programs.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Lila Sciences
@LilaSciences
Our CEO @GVMaltzahn opened "The Future of Autonomous Science" at @WHOSTP47 's Science: A New Golden Age Summit, alongside Liam Fedus, D.J. Kleinbaum, and moderated by @mkoeris of DARPA. Take a listen!
www.whitehouse.gov/science/This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
Come meet the researchers at @LilaSciences at Trellis in SF, 1 to 4pm today. @Techweek_ #sftechweek #techweek partiful.com/e/jxtyCAqL4xcVT5dw9PbT
(I won't be there myself but plenty of Lilans will be who will be happy to talk to you!) -
Kenneth Stanley reposted
Lila Sciences
@LilaSciences
Get ready #sftechweek is coming! On Oct 7 we'll be at @trellis.social for breakfast + a panel for women and non-binary people in science and AI/ML, then MEET LILA 1–4pm. Come join us! partiful.com/e/SD3GEgeitBZ8h44lESsF · partiful.com/e/jxtyCAqL4xcVT5dw9PbT #sftechweek
-
Kenneth Stanley reposted
Lila Sciences
@LilaSciences
Hello #sftechweek! Join us for breakfast and community conversations around women in AI for Science on Oct 8th. Spots are filling up! partiful.com/e/SD3GEgeitBZ8h44lESsF
-
Kenneth Stanley
@kenneth0stanley
Congrats to our CEO @GVMaltzahn ! “Visionary” is indeed a good term to describe @LilaSciences’s leadership.
Geoffrey von Maltzahn
@GVMaltzahn
Honored to be named one of @FastCompany 's Tech Visionaries of the Year, and grateful to share it with the team at @LilaSciences.
The line in the piece that means the most to me is about the people we build for: "Researchers can now greatly expand the scope of the problems they are attacking." Proud of the team making that real every day.
www.fastcompany.com/91612657/tech-visionaries-year-next-b…
#fctechawards -
Kenneth Stanley reposted
Lila Sciences
@LilaSciences
"Researchers can now greatly expand the scope of the problems they are attacking." @FastCompany on why @GVMaltzahn is one of 2026's Tech Visionaries of the Year. #fctechawards www.fastcompany.com/91612657/tech-visionaries-year-next-b…
-
Kenneth Stanley reposted
Geoffrey von Maltzahn
@GVMaltzahn
Honored to be named one of @FastCompany 's Tech Visionaries of the Year, and grateful to share it with the team at @LilaSciences.
The line in the piece that means the most to me is about the people we build for: "Researchers can now greatly expand the scope of the problems they are attacking." Proud of the team making that real every day.
www.fastcompany.com/91612657/tech-visionaries-year-next-b…
#fctechawards -
Kenneth Stanley
@kenneth0stanley
Excited and honored to be speaking at the ARC Prize Research Summit on October 23rd! With the next generation of ARC Prize focusing on open-ended discovery, this event is a great opportunity to discuss what it really means to succeed in creating truly open-ended AI.
ARC Prize
@arcprize
Announcing ARC Prize Research Summit 2026 Keynote Speaker
Kenneth Stanley
@kenneth0stanley is a pioneer in neuroevolution and open-ended AI, co-creator of NEAT and novelty search, and co-author of “Why Greatness Cannot Be Planned.” -
Kenneth Stanley reposted
Alon Albalak
@AlbalakAlon
@LilaSciences is hiring 🚀 If you're attending @COLM_conf, chat with us at our booth in the industry expo
I'll personally be at the booth from 2-4pm on tuesday, come by to say hi👋
Alon Albalak
@AlbalakAlon
Friends! Who's going to be at COLM next week? I'd love to catch up with old friends and meet up with new ones!
-
Kenneth Stanley reposted
Kanjun 🐙
@kanjun
What should the future of personal computing look like?
We built it 😈
Introducing Imbue Studio:This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
Congrats to @hardmaru @risi1979 @yujin_tang and Risto! Neuroevolution is where I got started in AI, and neuroevolution is still a playground for original ideas with neural networks. This book is worth reading!
hardmaru
@hardmaru
Our Neuroevolution textbook is finally in print!
Free online edition: neuroevolutionbook.com
Pre-order: a.co/d/06DpkH9h
I am incredibly grateful to my co-authors Sebastian Risi, Yujin Tang, and Risto Miikkulainen for making this happen. Neuroevolution is a subject very dear to my heart. It is the field that convinced me that nature has already figured out how to build intelligence: through evolution, collective behavior, and adaptation under constraints.
This idea, that intelligence emerges from evolution operating under constraints rather than unlimited resources, is what eventually led me to founding Sakana AI here in Japan.
The concepts in this book about open-ended creativity and self-organizing systems are exactly what we build at @SakanaAILabs. Our name and logo are inspired by schools of fish moving together, adapting as one. It is the core philosophy of what we build. This book captures the theoretical foundations of that belief. -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
It just became real 😀
Sebastian Risi
@risi1979
I’m beyond excited to announce our MIT Press book on Neuroevolution! An HTML version is now available for free on neuroevolutionbook.com, with a print edition coming out later in 2026.
Real intelligence is not static; it evolves. For decades, the field of neuroevolution has pursued this necessary adaptability. Our book chronicles its development, from early concepts to its modern integration with deep learning and reinforcement learning, exploring its potential for understanding the origins of intelligence and its real-world applications.
And the companion webpage is more than just a book site! It comes equipped with interactive demos, videos, exercises, and tutorials to allow everyone to experience neuroevolution in action. Check it out and let us know what you think!
It was a pleasure to work on this book over the last 4+ years with David (@hardmaru), Yujin (@yujin_tang), and Risto. We are incredibly proud of the result and look forward to celebrating! We hope to connect with many of you at NeurIPS.
We are very grateful to Melanie Mitchell (@MelMitchell1) who provided a fantastic foreword. To quote her: “The next big thing in AI is coming, and I suspect that neuroevolution will be a major part of it”. We think so too! -
Kenneth Stanley reposted
Andrew Beam
@AndrewLBeam
I'd like to share one of my favorite stories about a discovery we made at @LilaSciences.
We’ve been working for a while on catalysts for clean hydrogen. One day, our AI proposed chemistry that a scientist who’d studied this exact reaction for 12 years thought had to be wrong.
That composition became our strongest lead, sustaining more than 1,000 hours of oxygen evolution with little loss of catalytic performance.
The scientist is John Gregoire (known as JG to us), who left his Caltech research professorship to become our Chief Autonomous Science Officer, and has been working to build some of the world's most sophisticated automated labs at Lila.
Making clean hydrogen from water requires electricity to drive two reactions: one produces hydrogen, the other oxygen. The oxygen-producing half, the oxygen evolution reaction (OER), is particularly difficult.
In proton-exchange-membrane electrolyzers, the catalyst must drive that reaction efficiently while exposed to strong acid and high voltage. Most materials either corrode or require too much energy to serve as an effective catalyst. Iridium oxide is the industry standard, but iridium is one of the rarest metals on Earth. Depending on it creates a supply constraint on scaling this technology.
JG’s former group at Caltech spent 12 years exploring catalysts for this reaction. He knows how unforgiving these conditions are.
Early in this project, the scientists checked the AI’s suggestions closely, occasionally overruling them. The choices were mostly familiar and sensible. As the experiments progressed, the team stepped back, retaining safety review while letting the system choose what to try.
By the second campaign, which took just four weeks, it was producing results they hadn’t expected.
The system had started exploring palladium-based oxides. When JG and Rafael Gómez-Bombarelli, our physical sciences CSO and an MIT professor, saw the results, they thought the model had gone astray. These were combinations that JG would have told his grad students weren't even worth testing.
But the measurements were promising, so the team kept testing.
Across 2,942 catalysts spanning 53 material systems and 26 elements, the platform uncovered an unexpected family of palladium-based oxides with a promising combination of activity and durability. It screened around 240 catalysts a week, an order of magnitude faster than a standard lab by our team’s estimate.
The best contained tiny additions of indium and manganese, together accounting for about 0.09% of its metal atoms.
At 10 mA/cm² in 1 M sulfuric acid, it maintained an overpotential below 0.5 V for more than 1,000 hours and retained 96% of its palladium. Unmodified palladium oxide crossed that voltage threshold after only 200 hours.
Under the microscope, the team saw a needle-like structure that developed during operation and appears to be associated with the improved durability. We’re still working to understand exactly why it works.
Palladium is also a precious metal, but its supply base is much larger than iridium’s. These catalysts contain neither iridium nor ruthenium, potentially giving us another option.
Giving the AI control over which experiments to run was decisive. The system combined predictive models that learned from our experimental data with LLM reasoning that helped decide where to search next. It could pursue chemistry our scientists would have passed over, have the lab make and test it, and use the measurements to choose its next experiments. Each round of results changed where it looked next.
More than 90% of the screening workflow was automated, with scientists providing oversight and transferring samples between instruments.
These are laboratory results. There’s still substantial work ahead to establish performance under industrial conditions.
People ask when AI and automated labs will start accelerating science. For us, it’s already changing what gets tested and how quickly we learn.
We can make hundreds of materials a week, test them, and use the results to decide what to try next, including ideas our own experts would have passed over.
I want a lot more of this: AI helping us discover things we can actually use to make the world better.
Full story and preprint below 👇 -
Kenneth Stanley
@kenneth0stanley
Proud to be at the company where scientific advances that can improve our lives are being vastly accelerated.
Lila Sciences
@LilaSciences
Green hydrogen currently runs on iridium — one of Earth's rarest metals, a few tonnes mined per year. Our AI-directed lab screened 2,942 catalysts in 3 months and surfaced a palladium-based lead the field had written off, now past 1,000 hours of testing in acid.
-
Kenneth Stanley
@kenneth0stanley
Aren’t parrots already stochastic? What did putting the word “stochastic” in front do for the metaphor? People are complaining about how it undersold the power of LLMs, but no one is defending the poor parrots from the insinuation that they are deterministic.
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Our research and @Recursive_SI in the @nytimes today.
www.nytimes.com/2026/09/16/science/ai-recursive-self-impr… -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
I like this article in MIT Tech Review South Korea. It covered many important topics, and allowed me to describe what I feel like is a new type of RL that is only recently possible, yet powerful. Curious to hear what everyone thinks of my answers. jeffclune.com/media/shared/Jeff_Clune_Interview.pdf
-
Kenneth Stanley reposted
Greg Kyro
@Gregory_Kyro
I'm building Scientific Superintelligence at @LilaSciences so that we can cure all diseases
This requires self-play against reality, where an AI agent can formulate hypotheses, design and execute experiments, and refine its reasoning through empirical feedback
If you're interested in working together on this, DM me the coolest thing you've built or discovered -
Kenneth Stanley
@kenneth0stanley
Turning to open-ended innovation as the next frontier for AI makes a lot of sense, I commend @arcprize for moving in this direction, but I’m very curious how they will conceive a “benchmark” for “open-endedness,” two words that seem almost antithetical to each other. In fact, one likely reason that open-ended innovation has lagged behind other areas of AI is how fundamentally resistant it is to benchmarking.
Now that doesn’t necessarily mean there’s no hope for an imaginative approach. Attempts at measuring open-endedness go back to Bedau’s activity statistics in the field of artificial life, Several colleagues and I later introduced a measure called “ANNECS — Accumulated Number of Novel Environments Created and Solved” in our paper on Enhanced POET. That’s not an exhaustive list.
But there’s never been the kind of benchmark where you can just easily put any systems seamlessly head to head, and there are enormous pitfalls if you get wrong. After all, if “open-ended innovation” ends up equated to “solving a prescribed.hard problem in a creative way” then you risk actually rewarding the opposite of open-endedness, which needs to account for the fact that deciding the “problem” or objective is part of the job of the open-ended system itself. And also, perhaps even more prohibitively for benchmarking, that a key aspect of open-endedness is to be intelligent when you don’t have a defined objective or problem at all! How can that be benchmarked?
I still think it’s great that ARC Prize is bringing attention to this part of AI space, and I’d be happy to connect and exchange thoughts on how to get it right if that could be useful.
ARC Prize
@arcprize
ARC-AGI-4 will be a benchmark for autonomous open-ended innovation. It will continue our commitment to open-source, giving the research community a shared target for progress that benefits all of humanity.
Despite rapid model progress, humans still significantly outperform AI at open-ended invention. This is the meta-skill that unlocks progress across every field of technology.
Advanced AI capable of scientific innovation will lead to tremendous new technology, knowledge, and understanding. This is a positive-sum future. We are deeply committed to advancing it.
Open source is the foundation for that progress.
The knowledge behind frontier AI, not just the technology itself, should be broadly distributed among researchers, academics, and organizations. Any coordinated effort by the AI industry to reduce openness or concentrate access to frontier AI would undermine that positive-sum future. We are committed to advancing a future where everyone can contribute to and benefit from AI progress. -
Kenneth Stanley reposted
Kunal Jha
@kjha02
Can self-interested, self-improving, self-replicating agents learn to cooperate?
Our new paper, Tapes Together Strong, shows they can: when social behavior, computation, and reproduction share one energy budget, cooperation evolves from scratch.
arxiv.org/abs/2609.10817 🧵 -
Kenneth Stanley reposted
Hussain Nadim
@HNadim87
Long ago, philosophers, sadhus, and sufis all arrived at the same conclusion:
The destination is only as good as the journey.
Kenneth Stanley
@kenneth0stanley
There’s something really interesting about the field of mathematics coming to the insight that the path is more important than the destination, which is clearly related to the insight that Fractured Entangled Representation (FER) in neural networks results from taking a bad path to a high-performing destination. Massively scaled AI is teaching us something deep about the importance of the path you take as opposed to the place you end up. This theme will be increasingly powerful as it washes over every field.
Thomas Wolf
@Thom_Wolf
Some notes from the Fields Medal letter today (mathandai.org)
Generally saying that AI models riffling through solving these problems gives 1 bit of signal while killing the 99 other bits of conceptual understanding that mathematicians would have extracted from fighting with these problems -
Kenneth Stanley
@kenneth0stanley
There’s something really interesting about the field of mathematics coming to the insight that the path is more important than the destination, which is clearly related to the insight that Fractured Entangled Representation (FER) in neural networks results from taking a bad path to a high-performing destination. Massively scaled AI is teaching us something deep about the importance of the path you take as opposed to the place you end up. This theme will be increasingly powerful as it washes over every field.
Thomas Wolf
@Thom_Wolf
Some notes from the Fields Medal letter today (mathandai.org)
Generally saying that AI models riffling through solving these problems gives 1 bit of signal while killing the 99 other bits of conceptual understanding that mathematicians would have extracted from fighting with these problems -
Kenneth Stanley reposted
Alon Albalak
@AlbalakAlon
I am incredibly privileged to work with this remarkable team at LILA on such an important, exciting mission
I’m especially grateful to Ken for bringing us together around a philosophy that shapes what we pursue and how we approach discovery. Excited to see where it leads!
Lila Sciences
@LilaSciences
A batch of "junk" diamond crystals turned out to be exactly what the researchers needed. That instinct -- don't throw out the weird result -- is the philosophy behind Lila's Open-Endedness team led by @kenneth0stanley www.lila.ai/news/whats-a-science-startup-doing-with-an-op…
-
Kenneth Stanley
@kenneth0stanley
So proud of our new open-endedness team @LilaSciences! We'll be sharing work with the world soon. In the meantime, you can learn more about us in this team profile Lila just posted. Our bet is that open-
endedness is mission-critical to real scientific discovery. More👇
Lila Sciences
@LilaSciences
A batch of "junk" diamond crystals turned out to be exactly what the researchers needed. That instinct -- don't throw out the weird result -- is the philosophy behind Lila's Open-Endedness team led by @kenneth0stanley www.lila.ai/news/whats-a-science-startup-doing-with-an-op…
Kenneth Stanley
@kenneth0stanley
Our amazing team members you'll want to follow: @AlbalakAlon @Evo_Daveo @tehqin17 Sebastian Gabriel @joelbot3000 @ThomasXSheppard @_joelsimon Ivy Zhang
... and growing!
Kenneth Stanley
@kenneth0stanley
Follow Ivy at @_ivyzhang
-
Kenneth Stanley reposted
Marko Cvjetko
@marko_cvjetko
🧵 New paper: RL for discovering and controlling self-organizing phenomena.
We introduce CARL, a closed-loop agent that learns to create self-organizing patterns in Lenia, steer their behavior, and let humans guide them in real time.
Project website + interactive demos below ↓ -
Kenneth Stanley
@kenneth0stanley
What we’re witnessing right now in AI for math is a dramatic improvement in objective-driven problem-solving, but it does not speak to non-objective discovery. I have always maintained that objectives are achievable when they are one stepping stone away, which means that they are within reach if their necessary prerequisite stepping stones are already laid. We say as much in Why Greatness Cannot Be Planned.
Furthermore, I have identified “the stepping stepping already laid” as every invention and idea ever hatched over the course of human history. That is the frontier of civilization. But a massive challenge for every creative field is to make all the relevant stepping stones actually accessible to its practitioners. How can anyone be aware of all of human knowledge in their field? To the extent they cannot, some dots cannot be connected even if they’re already uncovered.
AI advances today have suddenly made the existing stepping stones vastly more accessible. It can see something close to “all” of them, which means it has the possibility of leaping to many more next stepping stones in the chain. Combined with a brute force-ish “try every known stepping stone under the sun”, that’s what we’re seeing in the pursuit of popular objectives.
But discovery and innovation are not only about finding the right existing stepping stones to solve your objective because for many problems the stepping stones are not already laid and also do not follow directly from existing stepping stones. If those problems are ever to be solved, they will require open-ended, non-objective search through spaces of ideas that are motivated not by solving a problem as an objective, but by being interesting in their own right. The advances we’re seeing today speak only faintly to that purpose, which is why open-endedness remains firmly at the frontier of AI. -
Kenneth Stanley
@kenneth0stanley
It was a pleasure to speak on stage with @scott_sage at @crane_vc’s Festival! Link to the recorded video is in the thread below 👇
Scott Sage
@scott_sage
I give two books away more than any other
- The Lion Tracker's Guide to Life by Boyd Varty
- Why Greatness Cannot be Planned by @kenneth0stanley
Both changed how I think about curiosity, which projects to pursue, not forcing outcomes...Relevant for building, investing, parenting...
It was an extreme privilege sitting down w Ken at Festival. Hope you enjoy the beautiful conversation and his wonderful insights. Thank you Ken!!👇 -
Kenneth Stanley
@kenneth0stanley
A good explanation of why open-endedness as a research direction within AI remains fertile ground for progress, even for AI in math, where the machines seem now to be racing ahead of humans.
Thomas Wolf
@Thom_Wolf
friends are regularly surprised when i say i don’t think math is (yet) “solved,” so in the wake of the ns result i figured i’d explain this take a bit more widely.
first, this is massively impressive and clearly an example of AI being, in some respects, far more powerful than the human mind. the team deserves huge praise for attempting and succeeding at this. i’d love to read a technical report on the project (one can always hope :)
but second, here again we ended up with a counterexample rather than a full proof: option C won the NS problem by proving the conjecture false (which was one valid way to solve the problem for sure)
notice a pattern in many of the recent frontier results in ai for math? a striking number involve counterexamples or finding a needle in a haystack.
now don’t get me wrong: this is extraordinarily hard and commendable. but it is only one aspect of mathematicians’ work.
Math is also about:
- finding deep, general mathematical understanding and explanations within proofs
- revisiting proven results to find more « elegant » proofs
- proposing new conjectures and hypotheses that might open « fruitful » directions
- taking the leap of faith of proposing entirely new, « exciting » research programs that may take decades to bear fruit
you may say these are simply further increments on the same intelligence scale, and perhaps point 2 above is already within the reach of current models. possibly
but you could also see this string of results as an extraordinarily powerful, massively parallel extension of search: explore the haystack, find the needle i.e. the counterexample that breaks the conjecture.
that would already be remarkable. but it would still leave open whether models understand the words I emphasized above: « elegant », « fruitful », « exciting ».
as tristan put it in his brief report: “the first llm generated proof levent sent me was the most horrendous i have ever read.”
despite their formidable technical abilities, models still seem to lack something mathematicians rely on constantly: a type of mathematical taste. the ability to distinguish a beautiful argument from a merely valid one, a fertile idea from a sterile one.
i’d be very curious to ask gpt or claude what they think of the result they just proved. do they find it elegant? if they ignore all the human noise about it on the web and history of math, do they appreciate the result and the path to it more than other proofs?
my suspicion is that, outside of the human knowledge that this is an important millennium prize problem, it may not 100% register it as fundamentally different from thousands of other proofs.
or, more intriguingly, perhaps the beauty they find in mathematics is actually alien to our own sense of mathematical beauty, some « horrendous » proof to us might be deeply satisfying to them…
all this to say that we might have significant progress to make in ai for math before declaring the field “solved,” (as too many are posting).
in the meantime, ai will be an extraordinary collaborator for mathematicians. i just hope access to these tools be as widely shared as possible (so that you don’t need a Leven or Sebastien working in a big lab to help you), and that companies stop using mathematics primarily as a demonstration of prowess in their private race. -
Kenneth Stanley reposted
Julian Togelius
@togelius
Big news! I'm leaving New York!
...but what on earth? Why? Where am I going? -
Kenneth Stanley
@kenneth0stanley
Proud to be working with @BenKompa at @LilaSciences , a well deserved honor!
Ben Kompa
@BenKompa
honored to be named to @techreview 's Innovators under 35 list!
the article below highlights the work we're doing @LilaSciences to build scientific superintelligence with @GVMaltzahn , @AndrewLBeam and an amazing team of too many folks to mention
we are working on scaled verification of AI hypotheses in the real world and are going full stack from post-training language models to running automated labs -
Kenneth Stanley
@kenneth0stanley
"One remaining area of important capability we haven't seen demonstrated yet by any model is open-ended invention and discovery."
Benchmarks keep going up. Open-endedness keeps moving up in importance.
Mike Knoop
@mikeknoop
GPT-6 Astra is the new SOTA on ARC-AGI-3
It's a qualitatively large leap towards AGI and the pace of progress is frankly surprising. That said, we lack evidence to call this AGI yet.
While we are still studying the human capability gaps, we believe open-ended invention is unsolved, and this will form the new basis for ARC-AGI-4.
Reminder that ARC v3 tests whether models can figure out how to efficiently make sense of unfamiliar environments and operate autonomously toward goals within them.
Summary: properly equipped Astra can now do this.
Astra direct model scored 66% at ~$500/game. Up from Sol 8%. This is apples-to-apples with all other ARC v3 scores we've verified to date. This score gives the best look at the general intelligence of base Astra.
Using an OpenAI-specific adapter (which is open source) we verified the first ~100% at $300/game. If you're using Astra via API I would prioritize migrating to /conversations endpoint with compaction enabled.
Direct model testing continues to carry important scientific interest towards AGI. But provider adapters are a better estimate of what you should expect day-to-day when using these models inside of products.
The action efficiency of Astra is also pretty incredible. With the provider adapter, Astra used 50% fewer actions on average than our human baseline (who both score 100%).
A surprise for us is that across all models, action efficiency on v3 is very bi-modal. Models either get the game or not. Weaker agent models aren't able to "brute force" their way to an inefficient understanding.
Tactically, our results support always using Astra's higher reasoning tiers when deploying agents into environments. It not only makes Astra better but is cheaper too (as a result of needing less turns).
The key technique Astra seems to leverage is on-the-fly symbolic world modeling. It develops a custom algebra and notation for the environment through interaction and uses this understanding to plan (all in token space).
This approach is similar to earlier harnesses we saw which created symbolic models through side-car executable programs to model the environment and plan.
Based on Astra results I now see clear paths to recursive self improvement via horizontal data scaling and continual learning where the learned state is information stored and managed outside of the model weights (text, programs, databases, etc).
I expect more and more "learning" to happen outside the model weights and this will be of growing interest to both researcher and AI product builders.
One remaining area of important capability we haven't seen demonstrated yet by any model is open-ended invention and discovery.
Humans clearly posses this capability and we've so far seen zero evidence that frontier AI -- even as powerful as Astra -- is capable of this feat.
Consider AI research 10 years ago vs today. Humans invented so much: transformer, self-supervised LLM pretraining, diffusion, RLHF, test time adaptation, ...
Society needs AI systems that not only can autonomously operate towards defined goals. We need AI systems help craft great futures that are inherently undefined.
I believe we in at a very tenuous spot. We now have extremely powerful automation system that capable of causing a lot of disruption.
Without the counterbalancing force of demonstrated invention by AI, I fear society will turn more and more against open frontier AI research and development -- trapping us in a very bad middle ground.
We must move as swiftly and with focus towards this important AI capability. And this has become the primary focus of our work on ARC-AGI. -
Kenneth Stanley reposted
Lila Sciences
@LilaSciences
What if a cancer therapy could go from idea to breakthrough result in 6 months instead of years? That's what our AI platform just helped make possible. www.lila.ai/news/inside-lilas-rna-discovery-engine
-
Kenneth Stanley reposted
Lila Sciences
@LilaSciences
Our CTO @AndrewLBeam delivering today’s keynote at #raysummit2026!
-
Kenneth Stanley reposted
Julian Togelius
@togelius
We got new cool stuff for ya. Especially if you're into PCG for games. Or genetic programming. Or agentic yada-yada, whatever it's called this month. What it is is an evolutionary loop that creates programs with LLM code writing, and abstracts a pool of reusable primitives.
Matthew Siper
@MatthewSiper
🧵 1/5
What happens when a language model doesn't just write code, but dynamically creates its own expert API mid-run?
Instead of evolving individual game levels, our new paper evolves entire generator programs using an LLM as the mutation and crossover operator. -
Kenneth Stanley reposted
Jakob Foerster
@j_foerst
TL;DR: This is one of the most important and exciting opportunities in AI on the planet - please read on.
The British Open-ended Learning & Discovery Lab is creating the perfect place for paradigm breaking AI research in the name of open-source and open-science. We have agency, we funding, we have unprecedented amounts of compute*, but WE NEED YOU!
..and we have created the dream job for you: The BOLD Fellow. This job combines a fast-moving, high agency, collaborative environment with full academic freedom and a salary that pays the bills.
Apply by noon UK time on the 15th of September for this once in a lifetime opportunity to shape the history of our field and of our planet:
my.corehr.com/pls/uoxrecruit/erq_jobspec_version_4.displa…
*by academic standards -
Kenneth Stanley reposted
Bert Chan
@BertChakovsky
at ALIFE 2026 @jeffclune talks about open-endedness and recursive self-improving AI
-
Kenneth Stanley
@kenneth0stanley
Congrats to @edwardfhughes @LouisKirschAI and the team at Inherent for new directions in open-endedness research!
Inherent
@inherent_labs
1/ Today, we introduce Faraday, a 27B-parameter AI Scientist that extends the capabilities of coding agents with a layer of scientific intuition. Trained via long-horizon RL, Faraday outperforms Claude Opus 4.8 and GPT-5.5 on the task of replicating research papers. 🧵
-
Kenneth Stanley
@kenneth0stanley
I wanted to share an experience about how difficult it is to automate “good writing” through LLMs. I’ve received many emails about AI from students over the years, going back long ago to around the time I shared source code for NEAT. These communications are often endearing. The joy of someone discovering a new frontier is a privilege to share. The “quality of writing” hardly matters - it’s the authenticity that is so compelling.
But now, in the last year or so, I’ve noticed a switch in many of these kinds of emails. Not all of them, but sometimes, they’re displays of polished genericism, like news releases about someone’s personal work, with words and phrases few young people would naturally choose.
And that gives me a surprising feeling of sadness. I can’t feel the person there anymore experiencing the rush of new ideas and possibilities. I’ll still try to respond, and I am not trying to wage a crusade against assisted writing, but they don’t realize that their true power was their own authentic words, and this new shortcut they think they’ve found has taken that away from them.
This scenario will not always be the case - sometimes we just want a generic clean exposition of a concept or idea. But when you’re engaging in interpersonal communication (or even art), you’ll never find a tool beyond your own mind that makes you seem more like your authentic self.
Paul Graham
@paulg
I was curious why LLMs have gotten so good at math and still aren't that good at writing, so I asked an expert. It's not because math is easier. It's because it has clear right and wrong answers and so is easier to train on. But they are coming for me next.
-
Kenneth Stanley reposted
Giuseppe Paolo
@_GPaolo
TerraLingua is live! And now open to the public! 🌍🤖
Create an autonomous AI agent, introduce it into a persistent world, and see what happens:
www.cognizant.com/us/en/ai-lab/terralingua
A thread 🧵 -
Kenneth Stanley reposted
Machine Learning Street Talk
@MLStreetTalk
SITUATION DETECTED:
We got invited to the @ndea pod to speak about creativity in AI. @bryanlanders was an amazing host!
Ndea
@ndea
On the pod: @MLStreetTalk's paper "Why Creativity Cannot Be Interpolated" with @ecsquendor + Dr. Jeremy Budd.
A deep dive into creativity, understanding, neuro-symbolic AI, human-AI collaboration, and how systems capable of extending their own phylogeny could become creative.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
Thanks to @BlackstoneAudio , the audiobook version of Why Greatness Cannot Be Planned is finally available! I'm glad it was finally possible to get this out!
Links 👇
Kenneth Stanley
@kenneth0stanley
Audible:
www.audible.com/pd/Why-Greatness-Cannot-Be-Planned-Audiob…
Kenneth Stanley
@kenneth0stanley
Downpour:
downpour.com/products/book-1fl9 -
Kenneth Stanley reposted
Sam Earle
@Smearle_RH
Proud to have received a Best Paper Award for our AI Picbreeder work at GECCO 2026 @GeccoConf in the Complex Systems track. Blog post here: pub.sakana.ai/picbreeder-vlm/.
(Along with an Outstanding Reviewer award in the Evolutionary Machine Learning track!🧎🏻♂️🙏🏻)
Sam Earle
@Smearle_RH
Our new work, The AI Picbreeder Experiment, explores the use of frontier models as drivers of synthetic open-endedness.
If we're serious about putting these things in the driver's seat of a new and automatic science, then we need to know what they're really made of in terms of the ability to create and discover through intuition. Giving shape to the formless, making decisions based on vibes, having "taste"—whatever you want to call it—Can they do it? Do they have the sauce?
Picbreeder, a website where human users collaborated to spontaneously evolve images, is just the sauce-bearing test we need. Here, images were represented as neural networks that could be bred and mutated, with humans playing the role of natural selectors. By design, this interface prohibits the creative baggage of premeditation, of having goals in advance, and demands the artist patiently follow the flow of the work and seize upon serendipitous opportunities when they arise. It's more like catching fish from a stream than drawing a picture. And yet, distributing their work across many sessions, and branching and remixing each other's creations, humans were ultimately able to bend these neural networks into all manner of interesting, evocative, and striking images.
So, can large vision language models do the same? On the blog, we've built an interactive archive viewer that allows visitors to walk through galleries of Picbreeder images created by both humans and AI, and judge for themselves.
Call us old fashioned, but we're pretty sure the human output has something special that the AI can't quite yet replicate. We design a number of evaluation metrics to get at this quality. We ask: "How visually different are the images in the archive? How much do they look like real things? How different are the things they look like?"
The numbers show the humans coming out on top. And looking at the AI-generated archives and lineages, we find traces of an anxious attachment to plans and objectives. Often, even when the AI makes an apparent creative leap—e.g. transforming an image of a hood ornament into a side view of a car—it really stays stuck in place in some broader semantic/thematic space. And that's to say nothing of the handful of archives littered almost entirely with top-down views of soda can pull tabs, or high frequency circular patterns that appear chaotic and uninteresting to us, but apparently scratch some perceptual itch in the agents.
And yet we're optimistic. Though the AI's output is less refined, its movement through the stream of images less graceful and vivacious than our own, what we have here is a plausible model organism of open-endedness. The agents indeed (re)discover distributions of novel and interesting images when left to their own devices. They display a keen eye (even sometimes discovering optical illusions that might slip by a casual glance from a human), and explore persistently under considerable creative constraints. This allows us to model factors that are consequential to such open-ended exploration; i.e. injecting noise into the agents' decision making process, playing with their memory, and seeding them with subtly distinct personalities—all of which can be beneficial in the right doses.
And there's something to be said for searching without objectives. Prior work shows that if we optimize Picbreeder's pattern-producing neural networks to resemble a particular image (say, a skull), these representations will be fractured (meddling with their internal weights will immediately explode the skull beyond recognition), while the same neural image found by humans via open-ended exploration is robust to such perturbations, and even shows meaningful variations across them (e.g. the jaw opening and closing). Our VLM agents also stumbled upon images of skulls. Their representations are not as neatly semantically factorized as those discovered by humans, but neither are they nearly as fractured as those discovered by optimization.
This suggests that if we want to have AI build the next generation of AI, then it will be crucial to let them attack this problem through aimless wandering. Without this freedom, future models will be brittle and myopic; with it, they will have developed a more thorough model of the world, and an improved capacity for the kind of creative insight that is so quietly fundamental to the most meaningful of human endeavors.
Sakana AI
@SakanaAILabs
The AI Picbreeder Experiment: Can AI agents be creative when nobody tells them what to create?
Blog: pub.sakana.ai/picbreeder-vlm
In our new #gecco2026 paper, "In Search of the Ingredients of Open-Endedness: Replicating Picbreeder with Large Vision-Language Models", in collaboration with MIT and NYU, we revisit Picbreeder, a lost website where people collaboratively evolved images without any predefined objective. Users simply selected images they found interesting, allowing unexpected forms such as faces, animals, vehicles, and skulls to emerge gradually across many generations and many different people.
We recreated this process using vision-language model agents. The agents explore a shared archive, choose images to branch from, evolve new candidates, publish their favorites, and evaluate the creations of other agents. There is no target image and no explicit definition of what counts as progress.
The results reveal both the promise and current limitations of AI-driven open-ended discovery.
Compared with humans, VLM agents tend to keep circling back to the same kinds of images and concepts. They repeatedly select similar parents, make smaller conceptual leaps, and often refine an existing idea rather than abandoning it in search of something genuinely unexpected.
However, introducing a diverse population of agent personalities substantially improves exploration. In some runs, diverse agent populations approached or matched the human archive on measures of semantic diversity and produced more balanced evolutionary trees.
We also find intriguing evidence that open-ended evolution can produce more robust representations. A skull evolved by the agents changes smoothly when its underlying neural representation is perturbed, less fractured than a skull directly optimized with gradient descent, although still less cleanly disentangled than one evolved collectively by humans.
But perhaps the most interesting result is the gap that remains.
Humans appear better at turning fortunate accidents into sustained creative discoveries: recognizing when something unexpected is worth pursuing, refining it, and then making a larger conceptual leap. The AI agents often notice interesting patterns too, but are more likely to become trapped in them.
We still do not fully understand what enables humans to navigate open-ended search in this way, or what ingredient(s) current AI systems are missing. For now, the results suggest that there remains something important about human creativity that AI agents have not yet learned to reproduce.
This paper will be presented at #gecco2026 and is nominated for a best paper award! Please check out the interactive blog and technical paper for more details!
Read our full paper: arxiv.org/abs/2605.23908 🐟 -
Kenneth Stanley reposted
Patrick OShaughnessy
@patrick_oshag
John Kim has raised over $70 billion in his 30-year career.
At General Catalyst, he raised several flagship funds and helped build the firm into one of the largest venture investors in the world.
Now he leads fundraising at Lila Sciences, a company building scientific superintelligence.
His book The Tao of Fundraising is the definitive guide to attracting capital, and this conversation is a masterclass in it.
We discuss:
- How General Catalyst built consensus
- Why money moves at the speed of trust
- Persuasion = desire minus fear
- The 3 laws of fundraising
- The secret of the GP/LP relationship
Enjoy!
TIMESTAMPS
0:00 Intro
1:00 Starting a Fundraise
10:40 How General Catalyst Built Consensus
16:56 The Three Laws of Fundraising
27:01 The Psychology of Every Sales Meeting
36:49 The Secretary of State Model
46:12 The Inner Game of FundraisingThis media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
Amazing to see that @SakanaAILabs recreated the entire Picbreeder website almost exactly as it once was!
pub.sakana.ai/picbreeder-vlm/breed/
@jsecretan and I have talked about resurrecting it in some form over the years, but I never thought I'd see it again this way! -
Kenneth Stanley
@kenneth0stanley
This is a beautiful and thought-provoking homage to Picbreeder from @SakanaAILabs, led by @Smearle_RH. Picbreeder is often misunderstood as a kind of toy. In reality, it was perhaps the first, maybe only, example of a truly open-ended artificial system that really took off. And by having an artifact like that in the world that was driven by human input, we have a rare comparison point to see what AI can do when unleashed in that same environment where humans once explored. Sakana's team really took that opportunity to heart here and as a result we get to see a unique glimpse of how the frontier of AI (VLMs) compares to humans in an open-ended, creative space.
Not surprisingly, VLMs come up somewhat short here compared to humans, though what they can do is still impressive. But more fundamentally I don't see this experiment mainly through the lens of "which is better." What I think it highlights is that as the pursuit of open-ended creativity through LLMs and AI becomes increasingly central to the field (AI for math! AI for science! AI for discovery/innovation/entrepreneurship! AI for art, literature, music!), it is essential that we have domains that are radically different from standard AI benchmarks, which are antithetical to questions of open-ended discovery.
Even when AI (impressively) solves a longstanding open problem in math, that is a different kind of impressiveness than open-ended discovery, wherein the AI is *not* told what it's meant to achieve. Picbreeder is one of the few, maybe only, genuinely open-ended spaces that humans have actually extensively explored through their own intrinsic curiosity (i.e. not asked to do it for a "study"). The opportunity that presents for us to learn about the creative and open-ended differences between humans and modern AI has been starkly neglected.
It's great to see someone finally take that opportunity seriously!
PS wow I'm also amazed they created a faithful reproduction of the original Picbreeder site that has been defunct now for years! You can go back in time and experience Picbreeder as it was: pub.sakana.ai/picbreeder-vlm/breed/
Sakana AI
@SakanaAILabs
The AI Picbreeder Experiment: Can AI agents be creative when nobody tells them what to create?
Blog: pub.sakana.ai/picbreeder-vlm
In our new #gecco2026 paper, "In Search of the Ingredients of Open-Endedness: Replicating Picbreeder with Large Vision-Language Models", in collaboration with MIT and NYU, we revisit Picbreeder, a lost website where people collaboratively evolved images without any predefined objective. Users simply selected images they found interesting, allowing unexpected forms such as faces, animals, vehicles, and skulls to emerge gradually across many generations and many different people.
We recreated this process using vision-language model agents. The agents explore a shared archive, choose images to branch from, evolve new candidates, publish their favorites, and evaluate the creations of other agents. There is no target image and no explicit definition of what counts as progress.
The results reveal both the promise and current limitations of AI-driven open-ended discovery.
Compared with humans, VLM agents tend to keep circling back to the same kinds of images and concepts. They repeatedly select similar parents, make smaller conceptual leaps, and often refine an existing idea rather than abandoning it in search of something genuinely unexpected.
However, introducing a diverse population of agent personalities substantially improves exploration. In some runs, diverse agent populations approached or matched the human archive on measures of semantic diversity and produced more balanced evolutionary trees.
We also find intriguing evidence that open-ended evolution can produce more robust representations. A skull evolved by the agents changes smoothly when its underlying neural representation is perturbed, less fractured than a skull directly optimized with gradient descent, although still less cleanly disentangled than one evolved collectively by humans.
But perhaps the most interesting result is the gap that remains.
Humans appear better at turning fortunate accidents into sustained creative discoveries: recognizing when something unexpected is worth pursuing, refining it, and then making a larger conceptual leap. The AI agents often notice interesting patterns too, but are more likely to become trapped in them.
We still do not fully understand what enables humans to navigate open-ended search in this way, or what ingredient(s) current AI systems are missing. For now, the results suggest that there remains something important about human creativity that AI agents have not yet learned to reproduce.
This paper will be presented at #gecco2026 and is nominated for a best paper award! Please check out the interactive blog and technical paper for more details!
Read our full paper: arxiv.org/abs/2605.23908 🐟 -
Kenneth Stanley
@kenneth0stanley
I’m proud to have been involved in this work from @ALife_Institute , which originated before I joined @LilaSciences . I’ve long believed that artificial life has the potential to benefit from scale just like AI has. After all, if you think the human brain or an LLM has impressive scale, wait until you look at the scale of evolution on Earth!
A big reason natural evolution produced such magic, in effect all of living nature, is that it has unlimited potential to benefit from scale. Kind of like what we see in AI today. So the fact that Alife has been relegated to minuscule simulators with relatively infinitesimal compute (compared to AI) probably means we haven’t seen any of what artificial evolution could do with a genuinely powerful substrate at scale.
Microcosmos is highly realistic physical Alife simulator with serious thought given to scalability. And the initial experiments (harnessing quality diversity and CPPNs) look really cool, hinting at what could be possible if Alife meets its deep learning moment!
Artificial Life Institute(一般社団法人 人工生命国際研究機構)
@ALife_Institute
What's been holding artificial life back from the open-ended complexity we see in nature?
We posit that the issue, in part, lies in our computational substrates, which are either physically grounded but too slow to evolve at scale, or scalable but disconnected from real physics.
Introducing Microcosmos: a scalable, physically grounded, end-to-end differentiable ALife engine where lifeforms are elastic filament chains swimming in a viscous fluid. Accepted at ALIFE 2026.
alife.institute/en/blog/microcosmos-release
Why filaments? Chains of connected units, from bacterial flagella to nematode bodies, are among the most ancient and ubiquitous structures in biology. We model each as an elastic rod coupled to the fluid, sensing the local flow as drag and pushing back on it in turn. Because the fluid is physically simulated, this coupling respects real physical constraints such as Purcell's scallop theorem.
Just as a chain of amino acids folds into a protein, a filament can fold into a target shape. Because the full simulation is end-to-end differentiable, we can learn these folds directly with gradient descent.
Microcosmos expresses a rich space of locomotion, from hand-designed swimmers like tadpoles and jellyfish to gaits discovered via quality-diversity search, including sinusoidal undulation, directional turning, and paddling with flippers.
And it scales. Runtime grows linearly with particle count, not quadratically like simulators built on pairwise interactions. That makes evolution at scale feasible.
We release Microcosmos as an open platform for the ALife community to build on. Our hope is to offer a substrate that is grounded enough to be credible, scalable enough to support evolutionary search, and rich enough to inspire.
Paper: arxiv.org/abs/2607.02954v1
Code: github.com/alife-institute/microcosmosThis media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
Many people over the years have asked me whether there will be an audiobook version of Why Greatness Cannot Be Planned. Finally I have an answer: it'll be here thanks to @BlackstoneAudio on July 21st! Preorders👇
Kenneth Stanley
@kenneth0stanley
Audible: www.audible.com/pd/Why-Greatness-Cannot-Be-Planned-Audiob…
Kenneth Stanley
@kenneth0stanley
Downpour: downpour.com/products/book-1fl9
-
Kenneth Stanley reposted
ALIFE Conference 2027
@ALifeConf
We're excited to start unveiling the keynote speakers for #alife2026!
First up: @jeffclune (@UBC), presenting:
"Open-Ended, Quality Diversity, and AI-Generating Algorithms in the Era of Foundation Models"
Join us in Waterloo, Canada, 17–21 Aug, 2026.
2026.alife.org -
Kenneth Stanley
@kenneth0stanley
Congrats to @CULLYAntoine and the new @bold_lab_ai ! Great new initiative and excellent labs joining together with connections to open-endedness!
Antoine Cully
@CULLYAntoine
Incredibly proud to be a co-director of @bold_lab_ai 🚀
Five outstanding labs, from three world-leading institutions, one shared mission — with the resources to deliver on it.
Let's build something in the UK that the entire world can benefit from. 👉 bold-lab.ai
Kenneth Stanley
@kenneth0stanley
And great choice of logo!
Antoine Cully
@CULLYAntoine
Wondering why our logo is a butterfly? 🦋
It's a tribute to "Why Greatness Cannot Be Planned: The Myth of the Objective" by @kenneth0stanley and @joelbot3000 — a book that inspires everything we do at BOLD. -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
First results from Recursive on AI that improves AI! 🚀 ✨ 📈The same general system produces state-of-the-art results on three different problems (@karpathy's NanoChat autoresearch, NanoGPT speedrun, and kernel optimization). It automates key parts of the science loop: ideation, implementation, & testing.🧵👇
Jeff Clune
@jeffclune
First results from Recursive on AI that improves AI! 🚀 ✨ 📈
Our automated AI research system incorporates principles from open-ended and AI-generating algorithms. 🧠 💡🌱🧬
It conducts key parts of the science loop: proposing ideas, implementing them, testing them, and picking the next ideas based on the data. 🔬 🔭 🧪⚗️
The same general system produces state-of-the-art results on three different problems (two on training language models, one on speeding up AI via kernel optimization).
• NanoChat Autoresearch: 1.3x faster to reach the same loss than the best solution produced by an entire community of humans+agents over months, and 1.8x faster than the initial hand-optimized solution
• NanoGPT Speedrun: 3% speedup of a very efficient solution produced by entire community of humans+agents over 2+ years
• GPU Kernel Optimization: 18% reduction in gap to theoretically optimal score on Nvidia’s SOL-ExecBench
These are early tests of our system. We’re very excited for what the future holds!
Post: www.recursive.com/articles/first-steps-toward-automated-a…
Great work by the amazing team at Recursive! -
Kenneth Stanley reposted
Akarsh Kumar
@akarshkumar0101
We never really knew how to train nonlinear RNNs well… BPTT struggled with vanishing grads (no long-range memory) and sequential rollout (hard to parallelizable).
What if instead an oracle told us the optimal memory state m_t at each step? Then the RNN could do one-step supervised learning on (m_t, x_{t+1}) → m_{t+1} labels.
We call this Supervised Memory Training (SMT): a replacement for BPTT that trains RNNs without unrolling them. SMT is time-parallelizable and solves vanishing gradients.
Website: akarshkumar.com/smt/
arXiv: arxiv.org/abs/2606.06479 -
Kenneth Stanley
@kenneth0stanley
The problem is deeper than what @paulg is highlighting. It’s that we’re losing signals of authenticity. Language (especially written) has never been a perfect window into what is “truly inside” us, but with AI it is rapidly losing signal. A fix for this would change the world.
Paul Graham
@paulg
I have never knowingly finished reading an email signed by a human but written by AI. It feels like being lied to, and who would stand for that?
-
Kenneth Stanley reposted
Lila Sciences
@LilaSciences
We're honored to be recognized on CNBC's 14th annual #disruptor50 list.
"Lila is building a platform with the potential to disrupt the very process of scientific discovery, and usher in a new era of rapid innovation across a broad set of industries."
cnb.cx/4wEjldG -
Kenneth Stanley reposted
Ben Kompa
@BenKompa
come meet the @LilaSciences team during techweek boston! link below - we are building the full stack for autonomous science from post-training language models at scale to robots conducting experiments in the real world
-
Kenneth Stanley reposted
Machine Learning Street Talk
@MLStreetTalk
Earlier this year we released long-form article on creativity, and I am delighted that Giuseppe Coccia made a YouTube explainer of the article, it's really good!
The basic idea in a single sentence is that creativity is (counterintuitively) about maximally respecting constraints, not "randomness" or "novelty", hence "creativity cannot be interpolated".
We also think of "understanding" as a structure of compositional, path-dependent constraints (like walls in a maze) and that creativity is antithetical to intelligence and agency (both of which require some kind of directed optimisation with partial knowledge, when transformative creativity is always about traversing into the space of unknown unknowns (inspired from @kenneth0stanley book).
Link below 👇 -
Kenneth Stanley
@kenneth0stanley
Congratulations to my friends at Recursive, a major new force in open-ended AI! And yet another signal that open-endedness is increasingly the north star for the future of the field!
Jeff Clune
@jeffclune
Thrilled to share that we founded Recursive to create AI that safely conducts experiments on how to improve itself in an open-ended process of endless, automated scientific discovery. As I wrote in my 2019 AI-generating algorithms paper, this will likely be the fastest path to superintelligence. Our work since has shown the power of this approach. Excited to scale up and improve upon ideas like the Darwin Gödel Machine, HyperAgents, ADAS, OMNI, ALMA, The AI Scientist, PromptBreeder, Rainbow Teaming, Automated Capability Discovery, and other work on open-ended and AI-generating algorithms. We’ve assembled a dream team of researchers and significant resources to pursue this vision. My amazing co-founders are pictured here, and we have an all-star team of founding members (we’re over 25 and growing).
Please join us if you are interested! Follow our progress @Recursive_SI -
Kenneth Stanley
@kenneth0stanley
Surreal to see this. These life-size displays sitting today at an exhibition in Venice were long ago small images on my screen during a single session of Picbreeder where I serendipitously evolved a car. Crazy journey for these images!
matdryhurst
@matdryhurst
Picbreeder by @kenneth0stanley et al
-
Kenneth Stanley reposted
Fellowship
@fellowshiptrust
Strange Rules opens today at Palazzo Diedo, Venice, curated by @matdryhurst, @hollyherndon, and @HUObrist with Adriana Rispoli.
The exhibition introduces Protocol Art, a field of practice that looks at the hidden systems shaping digital culture, algorithms, AI models, computer protocols, platforms, and technological infrastructures, and transforms them into artistic material.
🗓️ Today’s program:
5:30 PM → Book Launch: E.A.T. INDEX with Hans Ulrich Obrist and contributors
7:00–9:00 PM → Cocktail Reception -
Kenneth Stanley
@kenneth0stanley
One day you're researching how humans explore vast search spaces through an obscure system called Picbreeder. Then almost 20 years later, Picbreeder images are going on display at a major art world event in Venice. Hard to believe! Why Greatness Cannot Be Planned in its essence.
It's a good opportunity to credit the full founding Picbreeder team who worked with me on this project, starting with the man who had the original idea to make a crowd-sourced image breeding site, @jsecretan :
Jimmy Secretan, Nick Beato, David D'Ambrosio, Adam Campbell, and Adelein Rodriguez
Great work team, almost 20 years later still achieving unexpected things!
Thanks also to @matdryhurst for seeing something deeper in Picbreeder than just a bunch of old genetic art.
matdryhurst
@matdryhurst
Strange Rules opens May 4th @ Palazzo Diedo Venice
Curated with @HUObrist @hollyherndon & Adriana Rispoli
Opening 7pm, all are welcome
🪱 -
Kenneth Stanley reposted
Lila Sciences
@LilaSciences
🤖What if a robot that doesn't know it's trying to walk learns faster than one explicitly trained to walk?
Our SVP of Open-Endedness @kenneth0stanley joined @DuncanCJ_ to discuss why objectives are the enemy of greatness, building Scientific Superintelligence at LILA, and the future of creativity.
🎧Watch the full episode⬇️
Duncan CJ
@DuncanCJ_
New episode with @kenneth0stanley on Why Objectives Are the Enemy of Greatness.
What if the surest way to fail at something ambitious is to have a clear plan to achieve it?
What if a robot that doesn't know it's trying to walk learns faster than one explicitly trained to walk?
Kenneth Stanley is one of the most provocative minds in artificial intelligence. Former professor and team leader at @OpenAI, he co-authored the cult classic Why Greatness Cannot Be Planned—a manifesto that challenges everything we think we know about achievement. His research discovered something shocking: in AI experiments, robots seeking novelty learned to walk better than robots explicitly trained to walk. Over the last two years, Ken has taken these theories from the lab into the real world. As SVP of Open-Endedness at @LilaSciences, he's building scientific super intelligence—AI that autonomously discovers new biology and chemistry.
His work reveals a profound truth: the path to greatness is paved with stepping stones you can't predict, and the greatest discoveries come from following what's interesting, not what's planned.
⏰ Timestamps:
00:00 - Introduction: When Objectives Become the Enemy
03:01 - How AI Experiments Led to Social Critique
08:00 - The Picbreeder Experiment: Breeding Pictures Online
15:00 - A Car from an Alien Face (Deceptive Stepping Stones)
21:50 - It's Not Random: The Sharp Compass of Interestingness
25:36 - From AI to Life Lessons: Permission to Follow Curiosity
32:55 - The Spaceship That Looked Like a Mistake
42:19 - Getting Lost in St. Petersburg: The Collection Metaphor
44:59 - Stepping Stones That Looked Like Mistakes
49:48 - Advice for the Founder Feeling Hollow Despite Hitting Metrics
53:29 - Balancing Investors, Boards, and Open-Ended Exploration
1:03:12 - Will Humans Become Curators of AI Creativity?
Look up Duncan CJ on YouTube, Apple Podcasts, etc. Enjoy!This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Reads with Ravi
@readswithravi
5) Why Greatness Cannot Be Planned by Joel Lehman and Kenneth Stanley
This book challenges readers to consider life without a destination and discovery without a compass. -
Kenneth Stanley reposted
Sam Earle
@Smearle_RH
My PhD thesis defense will be here zoom.us/my/togelius tomorrow (Monday) at 9am EST. All are welcome! 🙂 Talk title: "Open-ended Learning via Procedural Content Generation in Video Games: Environment Substrates, Morphogenesis, and Designer-Player Loops". Come watch me make it make sense!
-
Kenneth Stanley reposted
Uljad@NeurIPS
@uljadb99
Natural evolution's open-endedness leads to beautiful, complex emergent structures and self-organizing behavior 🌱✨. Replicating this in silico is famously hard 💻. Our paper points to a promising direction by evolving populations of competing neural cellular automata with lifelike behavior 🧬🤖 #isambard
⚠️⚠️flashing lights, rapid cuts, or strobe effects in this thread! 🚨🚨
1/nThis media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Boris
@BorisMeinardus
🚨Why should one huge LLM know and solve everything? - No single human does, yet our civilization does endless innovation.
Introducing AC/DC - it continually coevolves a population of small expert LLMs that collectively outperform GPT-4o.
(ICLR 2026 w/ @SakanaAILabs) 👇🧵This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
The more you learn the easier it should be to learn more. The key word is easier. What could be more natural?
That’s the real puzzle of continual learning. Merely avoiding brain damage from accumulating additional knowledge is barely scratching the surface. -
Kenneth Stanley reposted
Robert Lange
@RobertTLange
“What you can imagine is naturally a function of what you can learn.”
Kenneth Stanley
@kenneth0stanley
Difficulty achieving continual learning is also a bad omen for creativity: what you can imagine is naturally a function of what you can learn. Both are mediated by the adjacent possible to the same internal representations! Contorted algorithms (or the absence of clean options) for what should be simple and straightforward continual learning are therefore a hint that the large models they serve are creatively barren.
That explains why something that is close to “knowing everything” and often competitive with the abilities of experts can still produce fewer breakthroughs than you would expect from a human with similarly astounding knowledge and expertise. -
Kenneth Stanley
@kenneth0stanley
Difficulty achieving continual learning is also a bad omen for creativity: what you can imagine is naturally a function of what you can learn. Both are mediated by the adjacent possible to the same internal representations! Contorted algorithms (or the absence of clean options) for what should be simple and straightforward continual learning are therefore a hint that the large models they serve are creatively barren.
That explains why something that is close to “knowing everything” and often competitive with the abilities of experts can still produce fewer breakthroughs than you would expect from a human with similarly astounding knowledge and expertise. -
Kenneth Stanley reposted
Lila Sciences
@LilaSciences
We're in @Nature's latest Technology Feature on the 'self-driving' lab revolution. John Gregoire, our Chief Autonomous Science Officer, shares more on building our AI Science Factory and where we're headed.
nature
@Nature
AI-powered robotic tools are muscling in on tasks typically done by humans. What does the future hold?
go.nature.com/3PGxG8r -
Kenneth Stanley reposted
Kishen Patel
@quichenpatel
Finally got around to reading Why Greatness Cannot Be Planned by @kenneth0stanley and @joelbot3000 .
I think the book encapsulates something I've felt intuitively for awhile. That the best outcomes rarely come from intense optimization, but rather from following interesting threads.
Applies to investing, career decisions, research, building companies, really anything where the path to something great is nonlinear. The most valuable bets, on yourself or others, almost never fit neatly into a thesis. Highly recommend! -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Very excited to share this work with all of you! It is the next step in our line of work on the Darwin Gödel Machine and the Automated Design of Agentic Systems, and more generally in open-endedness, AI-generating Algorithms, and recursive self-improvement. Great work @jennyzhangzt! 🚀🚀🚀
Jenny Zhang
@jennyzhangzt
Introducing Hyperagents: an AI system that not only improves at solving tasks, but also improves how it improves itself.
The Darwin Gödel Machine (DGM) demonstrated that open-ended self-improvement is possible by iteratively generating and evaluating improved agents, yet it relies on a key assumption: that improvements in task performance (e.g., coding ability) translate into improvements in the self-improvement process itself. This alignment holds in coding, where both evaluation and modification are expressed in the same domain, but breaks down more generally. As a result, prior systems remain constrained by fixed, handcrafted meta-level procedures that do not themselves evolve.
We introduce Hyperagents – self-referential agents that can modify both their task-solving behavior and the process that generates future improvements. This enables what we call metacognitive self-modification: learning not just to perform better, but to improve at improving.
We instantiate this framework as DGM-Hyperagents (DGM-H), an extension of the DGM in which both task-solving behavior and the self-improvement procedure are editable and subject to evolution. Across diverse domains (coding, paper review, robotics reward design, and Olympiad-level math solution grading), hyperagents enable continuous performance improvements over time and outperform baselines without self-improvement or open-ended exploration, as well as prior self-improving systems (including DGM). DGM-H also improves the process by which new agents are generated (e.g. persistent memory, performance tracking), and these meta-level improvements transfer across domains and accumulate across runs.
This work was done during my internship at Meta (@AIatMeta), in collaboration with Bingchen Zhao (@BingchenZhao), Wannan Yang (@winnieyangwn), Jakob Foerster (@j_foerst), Jeff Clune (@jeffclune), Minqi Jiang (@MinqiJiang), Sam Devlin (@smdvln), and Tatiana Shavrina (@rybolos). -
Kenneth Stanley reposted
Lila Sciences
@LilaSciences
We're honored to be named one of Fast Company's Most Innovative Companies and in the top ten most innovative artificial intelligence companies of 2026!
Interested in joining our team? Come build the future of science with us: www.lila.ai/open-roles
#fcmostinnovative #hiring -
Kenneth Stanley
@kenneth0stanley
Job opportunity: the Open-Endedness Team at @LilaSciences is seeking a talented research engineer who would enjoy applying your wizard-like technical skills to genuinely deep and fundamental AI research into the algorithmic basis of creativity and innovation.
This role is about radical innovation at the research frontier and 10xing the capabilities of research scientists running world-changing experiments (we're not talking about “business solutions”): data pipelines, large models, distributed training, optimizing not just for current algorithms, but for exotic new approaches as well. We want our algorithms operating as efficiently as possible and scaling as seamlessly as possible as hardware resources expand. If that sounds exciting, apply below👇
Kenneth Stanley
@kenneth0stanley
When you apply let us know you saw my post here on X: job-boards.greenhouse.io/lilasciences/jobs/4184738009
-
Kenneth Stanley
@kenneth0stanley
Terrence Tao’s arguments for the value of serendipity and the counterintuitive benefits of inefficiency align perfectly with Why Greatness Cannot Be Planned. Too much emphasis today on optimization, getting the answer right, and getting exactly what we asked for.
Dwarkesh Patel
@dwarkesh_sp
Terence Tao spent a year at the Institute for Advanced Study - no teaching, no random events of committees, just unlimited time to think. But after a few months, he ran out of ideas.
Terence thinks that mathematicians and scientists need a certain level of randomness and inefficiency to come up with new ideas.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Duncan CJ
@DuncanCJ_
New episode with @kenneth0stanley on Why Objectives Are the Enemy of Greatness.
What if the surest way to fail at something ambitious is to have a clear plan to achieve it?
What if a robot that doesn't know it's trying to walk learns faster than one explicitly trained to walk?
Kenneth Stanley is one of the most provocative minds in artificial intelligence. Former professor and team leader at @OpenAI, he co-authored the cult classic Why Greatness Cannot Be Planned—a manifesto that challenges everything we think we know about achievement. His research discovered something shocking: in AI experiments, robots seeking novelty learned to walk better than robots explicitly trained to walk. Over the last two years, Ken has taken these theories from the lab into the real world. As SVP of Open-Endedness at @LilaSciences, he's building scientific super intelligence—AI that autonomously discovers new biology and chemistry.
His work reveals a profound truth: the path to greatness is paved with stepping stones you can't predict, and the greatest discoveries come from following what's interesting, not what's planned.
⏰ Timestamps:
00:00 - Introduction: When Objectives Become the Enemy
03:01 - How AI Experiments Led to Social Critique
08:00 - The Picbreeder Experiment: Breeding Pictures Online
15:00 - A Car from an Alien Face (Deceptive Stepping Stones)
21:50 - It's Not Random: The Sharp Compass of Interestingness
25:36 - From AI to Life Lessons: Permission to Follow Curiosity
32:55 - The Spaceship That Looked Like a Mistake
42:19 - Getting Lost in St. Petersburg: The Collection Metaphor
44:59 - Stepping Stones That Looked Like Mistakes
49:48 - Advice for the Founder Feeling Hollow Despite Hitting Metrics
53:29 - Balancing Investors, Boards, and Open-Ended Exploration
1:03:12 - Will Humans Become Curators of AI Creativity?
Look up Duncan CJ on YouTube, Apple Podcasts, etc. Enjoy!This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Giuseppe Paolo
@_GPaolo
What happens when AI agents are left to live (and die) together in a shared world?
We’ve been exploring this at the @cognizant AI Lab — and they started forming something that looks like a society. -
Kenneth Stanley
@kenneth0stanley
If AI will soon match any human cognitive skill, then enhancing your “AI skills” (or whatever similar meme) will not be a moat because using AI is itself a cognitive skill. So where’s your edge? The only thing you really have over AGI is your novelty: AGI can never be you.
You have 100 trillion connections in your brain. That’s a lot. No AI will ever precisely replicate those parameters. The training data isn’t there for AI to vacuum up because you are the only entity ever to live your life, and the only one who ever will.
The question is whether the sum and total of all that experience yields a novel perspective, where the value is in its uniqueness. Even today those who make a living off their perceived novelty tend to be the most successful. We anticipate a novel (yet often internally consistent) take from a public figure or leader or artist or intellectual we like or respect. Uniqueness and novelty will retain their edge in a post-AGI world because there are virtually infinite possible 100-trillion parameter minds, and even the largest model theoretically conceivable can never capture that whole distribution.
At the same time, the once-sterling premium of those skills that no longer make us unique is sinking. Expertise that once distinguished people, like how to code, is losing its edge. But the tricky part is that new skills, like “using AI effectively” are equally vulnerable. All of it just takes intelligence, and that’s the thing that’s being automated. Seeking some new “safe” skillset is a looming adventure in frustrating futility.
But what’s still left is your unique perspective. Novelty. No one and nothing can see the world through your eyes. But you have to nurture that uniqueness. Post-AGI, being like everyone else would be the real danger. -
Kenneth Stanley reposted
Josh Bongard
@DoctorJosh
Lecture 15 of Evolutionary Robotics course:
NEAT and HyperNEAT.
youtu.be/uDIWawd9FF8 -
Kenneth Stanley
@kenneth0stanley
Curious to see the beautiful new @LilaSciences office in SF, hang out with me, the open-endedness team, and other great researchers at Lila? Register here to swing by our SF Open House on the evening of March 18th: luma.com/LilainSF
-
Kenneth Stanley reposted
Julian Togelius
@togelius
How can it be that modern LLMs are so bad at playing games? Aren't they supposed to be generally intelligent? Honestly, they are better at coding games than playing them. Maybe programming is just a particular type of game? Our new position paper tackles these questions. (1/n)
-
Kenneth Stanley
@kenneth0stanley
It's interesting that we seek evidence of whether something is AGI in terms of whether a single model ("it") can do something like discover general relativity. But such discoveries are population-level feats, even if an Einstein is the final generator. The reason we could get one person in 1915 who could do that is that there were almost 2 billion diverse variations on the human mind at the time to choose from. To invest astronomical compute into a *single* model AI that could reliably do what Einstein did given knowledge up to 1911 is a vastly different problem than training 2 billion diverse models and finding the one (or it finding itself) with the right problem alignment. Humanity offers no precedent for cooking up a single model that can do anything the very best human can do in any field, though the subtlety of the distinction is easily overlooked.
Rohan Paul
@rohanpaul_ai
Demis Hassabis’s “Einstein test” for defining AGI:
Train a model on all human knowledge but cut it off at 1911, then see if it can independently discover general relativity (as Einstein did by 1915);
if yes, it’s AGI.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
When people talk about a human edge versus AI in "taste," the issue is not taste with respect to familiar things, but rather taste with respect to novel things like nothing anyone has ever seen or experienced before. AI may catch up, but humans at present maintain an edge here.
The ability to know that something completely out of left field will be seen as beautiful or cool or interesting or thought-provoking or the next big thing remains unconquered.
Nan Yu
@thenanyu
I hate to break this to everyone, but you probably don't have better taste than the AI.
If you ask any leading model about a product decision with guidance: "don't tell me to ask users; just reason through it yourself." It will give you a better answer than 90% of PMs.
A lot of people are afraid to do this because they suspect that the answers will actually be very good. There are plenty of other distinctly human things that we can contribute, but "having better taste" isn't one of them. -
Kenneth Stanley reposted
Machine Learning Street Talk
@MLStreetTalk
New high-effort article "Why Creativity Cannot Be Interpolated" co-written with Dr. Jeremy Michael Budd. Yes the name is a pun on the famous book by @kenneth0stanley!
The counterintuitive thesis (corollary of Kenneth's research):
- Intelligence and agency are orthogonal to creativity - and sometimes actively hostile to it.
- Genuine creativity is impossible without deep understanding and creativity without understanding is "slop".
The strangest property of LLMs: within a single frame they seem to comprehend so deeply, yet they possess no perspective of their own. Like the blind men and elephant parable, each report is accurate, yet none integrates. We call this "frame-dependent" understanding, and it will change how you think about AI creativity.
We started writing this 2 years ago, and this is our distilled understanding of AI creativity in 2026. -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Can AI agents design better memory mechanisms for themselves?
Introducing Learning to Continually Learn via Meta-learning Memory Designs. A meta agent automatically designs memory mechanisms, including what info to store, how to retrieve it, and how to update it, enabling agentic systems to continually learn across diverse domains. Led by @yimingxiong_ with @shengranhu 🧵👇 1/ -
Kenneth Stanley reposted
Josh Bongard
@DoctorJosh
I'm teaching my evolutionary robotics course again, for those who want to follow along.
Lecture 1: why make robots?
youtu.be/FjNNFRDvQgs?t=1659 -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Introducing Digital Red Queen 🟥👑
Remember Core War? Core War is a classic competitive programming game where tiny assembly-like programs (“warriors”) battle for control of a shared virtual computer. Code and data share the same memory, so warriors freely self-modify, self-replicate, and attack opponents by corrupting their instructions, making the environment chaotic and Turing-complete.
We let LLMs drive an evolutionary arms race in this sandbox using a continual self-play algorithm we call Digital Red Queen. Each new warrior must survive against an ever-growing lineage of past opponents. Over time, this produces increasingly robust programs and reveals convergent evolution! -
Kenneth Stanley
@kenneth0stanley
A question raised by the “platonic representation hypothesis” that I think I haven’t seen discussed: What if a tendency to converge to nearly identical representations would actually hint at underlying pathology in our toolset rather than signal virtue?
The fact that two people can both be educated to the point of world class expertise in the same discipline and yet one might still sharply diverge from the other’s model of the world is arguably not a bug but the very essence of the enduring human advantage in creative endeavors. After all, the ability to see the world anew despite an analogous set of inputs is the quintessential fuel for innovation itself.
So should we be excited if disparate models seem to converge to the same tired world models the more data we shovel into them, or should we be worried?
(Disclaimer: @phillip_isola et al’s initial paper on the platonic representation hypothesis is seminal, raising timeless provocative questions. Without it, I would not be raising this question myself.)
Quanta Magazine
@QuantaMagazine
As AI models grow more powerful, they appear to be converging on how they internally represent reality. @benbenbrubaker reports: www.quantamagazine.org/distinct-ai-models-seem-to-converg…
-
Kenneth Stanley
@kenneth0stanley
Lux is a great firm. This is a great opportunity for aspiring entrepreneurs and for the world.
Lux Capital
@Lux_Capital
We started Lux in 2000 with a simple conviction: the biggest opportunities lie at the frontier of science and technology that others find too hard, too early, or too confusing. Twenty-five years later, that conviction hasn't changed. What has changed is that the world has caught on.
Now compute, automation, and biology have matured into scalable engineering disciplines, and what was once fringe "deep tech" has become essential infrastructure. New talent, new tools, and new industrial capacity now exist to turn the most cutting-edge technology across the physical, computational, and life sciences into enduring businesses.
For these businesses and founders to become category-defining winners, they need meaningful capital and conviction. That is why we are especially excited to announce our largest fund to date, Lux Ventures IX, a $1.5B commitment to investing in the people turning sci-fi into sci-fact.
Today entire sectors of aerospace, biotech, defense, industrials, transportation and beyond are being reinvented by a new generation of brilliantly ambitious, often irreverent, scientists and engineers. And while markets have changed and capital reshapes around fewer companies and larger checks, our focus on the craft remains constant.
That focus matters because science doesn’t scale itself. The path from a result in the lab, only known to a few, to something durable the entire world will use requires a different kind of partnership. One that deploys the first $100K and the last $100M, that stays through the full arc of company-building, that mobilizes a powerful network of kinetic like-minded founders, and that treats capital as a tool rather than the product.
Forty-four people at Lux now manage $7B, still operating with the same discipline and purpose we had on day one. We find and fund founders others have overlooked, back conviction over consensus, and build alongside these rebels of science and tech at every stage.
We’re grateful to our valued LPs who've partnered with us through multiple cycles, and to the founders who trust us with their life's work. The mission continues, with even greater intensity and momentum.
www.luxcapital.com/news/announcing-lux-ventures-ix -
Kenneth Stanley reposted
Lux Capital
@Lux_Capital
We started Lux in 2000 with a simple conviction: the biggest opportunities lie at the frontier of science and technology that others find too hard, too early, or too confusing. Twenty-five years later, that conviction hasn't changed. What has changed is that the world has caught on.
Now compute, automation, and biology have matured into scalable engineering disciplines, and what was once fringe "deep tech" has become essential infrastructure. New talent, new tools, and new industrial capacity now exist to turn the most cutting-edge technology across the physical, computational, and life sciences into enduring businesses.
For these businesses and founders to become category-defining winners, they need meaningful capital and conviction. That is why we are especially excited to announce our largest fund to date, Lux Ventures IX, a $1.5B commitment to investing in the people turning sci-fi into sci-fact.
Today entire sectors of aerospace, biotech, defense, industrials, transportation and beyond are being reinvented by a new generation of brilliantly ambitious, often irreverent, scientists and engineers. And while markets have changed and capital reshapes around fewer companies and larger checks, our focus on the craft remains constant.
That focus matters because science doesn’t scale itself. The path from a result in the lab, only known to a few, to something durable the entire world will use requires a different kind of partnership. One that deploys the first $100K and the last $100M, that stays through the full arc of company-building, that mobilizes a powerful network of kinetic like-minded founders, and that treats capital as a tool rather than the product.
Forty-four people at Lux now manage $7B, still operating with the same discipline and purpose we had on day one. We find and fund founders others have overlooked, back conviction over consensus, and build alongside these rebels of science and tech at every stage.
We’re grateful to our valued LPs who've partnered with us through multiple cycles, and to the founders who trust us with their life's work. The mission continues, with even greater intensity and momentum.
www.luxcapital.com/news/announcing-lux-ventures-ix -
Kenneth Stanley reposted
Itai Yanai
@ItaiYanai
Because you won't get there [your goal] unless you stop caring about getting there. And so the only way to eventually solve the problem is to stop caring about the problem, which is what we meant when we said abandoning objectives – @kenneth0stanley on the Night Science Podcast
-
Kenneth Stanley
@kenneth0stanley
We had a very interesting discussion on this @nightsciencepod about the origins of scientific innovation and discovery. Thanks to @ItaiYanai
and @MartinJLercher for having me on!
The Night Science Podcast
@nightsciencepod
🔥New episode of the Night Science Podcast came out! Kenneth Stanley tells us how "the only way to eventually solve the problem is to stop caring about the problem, which is what we meant when we said abandoning objectives." @kenneth0stanley on Why Greatness Cannot Be Planned.
-
Kenneth Stanley reposted
The Night Science Podcast
@nightsciencepod
🔥New episode of the Night Science Podcast came out! Kenneth Stanley tells us how "the only way to eventually solve the problem is to stop caring about the problem, which is what we meant when we said abandoning objectives." @kenneth0stanley on Why Greatness Cannot Be Planned.
-
Kenneth Stanley reposted
Reads with Ravi
@readswithravi
My top 12 favorite books of 2025:
One lesson from each book 🧵 -
Kenneth Stanley
@kenneth0stanley
So much equivocation about AGI these days boils down to pillars of open-endedness: discovery, invention, creativity, self-improvement, research, diversity of thought, long horizons, continual learning. Open-endedness has always been destined to be the climax of the AI saga.
-
Kenneth Stanley reposted
Lila Sciences
@LilaSciences
➡️@FoxNewsSunday calls on @AndrewLBeam’s expertise, as Chief Technology Officer at Lila Sciences and a faculty member at Harvard University, to discuss the true impact of AI and what it could mean for human health.
🔗 Watch full episode and catch Andy’s segment at 30:04: -
Kenneth Stanley reposted
Richard C. Suwandi
@richardcsuwandi
2026 will be the year of open-endedness!
Alex Heath
@alexeheath
I asked NeurLPS attendees for their takeaways from the world's largest gathering of AI researchers
featuring @andykonwinski, @Thom_Wolf, @tszzl, @DynamicWebPaige, @natolambert, @ml_angelopoulos and more: sourcesnews.substack.com/p/googles-rise-rl-mania-and-a-party -
Kenneth Stanley
@kenneth0stanley
Natural evolution is such a neglected intuition pump for interrogating assumptions about creative intelligence. No world model, no predictions, no chain of thought, no beliefs, no organism an objective of the process, and phenotypic variation NOT simply random. Yet continual prodigious creativity and world class invention often beyond anything humans have achieved at literally biblical levels for over a billion years. Its radical lessons are ignored in discussions of automated invention and discovery in AI at our peril.
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
The usual suspects
-
Kenneth Stanley
@kenneth0stanley
Mechanistic/pragmatic interpretability should focus less on how networks “work” and more on measuring, detecting, and visualizing the degree of fractured entangled representation (FER) in large networks.
-
Kenneth Stanley
@kenneth0stanley
With all the talk about interpretability and alignment recently, one consideration missing from the conversation is the risk of paradigm shift rendering large swaths of insight on current architectures with current training conventions obsolete. Whatever ends up achieving aspirations such as continual learning or overcoming FER (fractured entangled representation) could upend broad assumptions upon which current and ongoing insights into interpretability and alignment are predicated. The mechanisms underlying behaviors like deception or alignment could change radically under such new paradigms.
That's why I think it's a mistake to focus exclusively on how contemporary models work and how they can be made to behave in certain ways. While such research is unequivocally valuable and should of course continue, it needs to be complemented by trying to understand what these models don't do well and why they don't do it. That way, interpretability can itself be a catalyst for the paradigm shift rather than just a bystander.
If you focus too much on studying how something works, you run the risk of assuming it actually works. -
Kenneth Stanley reposted
Lila Sciences
@LilaSciences
Grateful for another year of growth and the incredible team powering Lila’s mission. This is only the beginning. Interested in building the future of science with us? Explore roles at Lila: www.lila.ai/open-roles
#thanksgiving #hiring -
Kenneth Stanley
@kenneth0stanley
Reflecting the gut feeling of many, Ilya says “something important” is missing from current AI models. But what is the concrete nature of this chasm? One candidate: the difference between fractured entangled representation (FER) and unified factored representation (UFR).
Ilya Sutskever
@ilyasut
One point I made that didn’t come across:
- Scaling the current thing will keep leading to improvements. In particular, it won’t stall.
- But something important will continue to be missing.
Haider.
@haider1
here are the most important points from today's ilya sutskever podcast:
- superintelligence in 5-20 years
- current scaling will stall hard; we're back to real research
- superintelligence = super-fast continual learner, not finished oracle
- models generalize 100x worse than humans, the biggest AGI blocker
- need completely new ML paradigm (i have ideas, can't share rn)
- AI impact will hit hard, but only after economic diffusion
- breakthroughs historically needed almost no compute
- SSI has enough focused research compute to win
- current RL already eats more compute than pre-training -
Kenneth Stanley reposted
Bidipta Sarkar
@bidiptas13
Introducing 🥚EGGROLL 🥚(Evolution Guided General Optimization via Low-rank Learning)! 🚀 Scaling backprop-free Evolution Strategies (ES) for billion-parameter models at large population sizes
⚡100x Training Throughput
🎯Fast Convergence
🔢Pure Int8 Pretraining of RNN LLMs -
Kenneth Stanley
@kenneth0stanley
Nice depiction of paths to good (UFR) and bad (FER) representation here by @IntuitMachine !
Carlos E. Perez
@IntuitMachine
-
Kenneth Stanley reposted
Sebastian Risi
@risi1979
I’m beyond excited to announce our MIT Press book on Neuroevolution! An HTML version is now available for free on neuroevolutionbook.com, with a print edition coming out later in 2026.
Real intelligence is not static; it evolves. For decades, the field of neuroevolution has pursued this necessary adaptability. Our book chronicles its development, from early concepts to its modern integration with deep learning and reinforcement learning, exploring its potential for understanding the origins of intelligence and its real-world applications.
And the companion webpage is more than just a book site! It comes equipped with interactive demos, videos, exercises, and tutorials to allow everyone to experience neuroevolution in action. Check it out and let us know what you think!
It was a pleasure to work on this book over the last 4+ years with David (@hardmaru), Yujin (@yujin_tang), and Risto. We are incredibly proud of the result and look forward to celebrating! We hope to connect with many of you at NeurIPS.
We are very grateful to Melanie Mitchell (@MelMitchell1) who provided a fantastic foreword. To quote her: “The next big thing in AI is coming, and I suspect that neuroevolution will be a major part of it”. We think so too! -
Kenneth Stanley
@kenneth0stanley
I spent many years working in the field of neuroevolution. What could be more fascinating than the idea of evolving brains? Now finally there is a definitive text for the field to help future generations explore further. Congrats to @hardmaru @risi1979 @yujin_tang and Risto!
hardmaru
@hardmaru
Excited to announce our MIT Press book “Neuroevolution: Harnessing Creativity in AI Agent Design” by Sebastian Risi (@risi1979), Yujin Tang (@yujin_tang), Risto Miikkulainen, and myself.
We explore decades of work on evolving intelligent agents and shows how neuroevolution can drive creativity in deep learning, RL, LLMs and AI Agents!
📖 Free open-access edition: neuroevolutionbook.com
In addition to our own works, this video features work by Jürgen Schmidhuber (@SchmidhuberAI), Seth Bling (@SethBling), Igor Karpov, Jacob Schrum, Yulu Gan (@yule_gan), Ken Stanley (@kenneth0stanley), Joel Lehman (@joelbot3000), Jeff Clune (@jeffclune), Nick Cheney (@CheneyLab), Richard Song (@XingyouSong), Chelsea Finn (@chelseabfinn), Julian Togelius (@togelius), Sam Earle (@Smearle_RH), Hod Lipson (@hodlipson), and Jean-Baptiste Mouret (@jb_mouret).This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
hardmaru
@hardmaru
Excited to announce our MIT Press book “Neuroevolution: Harnessing Creativity in AI Agent Design” by Sebastian Risi (@risi1979), Yujin Tang (@yujin_tang), Risto Miikkulainen, and myself.
We explore decades of work on evolving intelligent agents and shows how neuroevolution can drive creativity in deep learning, RL, LLMs and AI Agents!
📖 Free open-access edition: neuroevolutionbook.com
In addition to our own works, this video features work by Jürgen Schmidhuber (@SchmidhuberAI), Seth Bling (@SethBling), Igor Karpov, Jacob Schrum, Yulu Gan (@yule_gan), Ken Stanley (@kenneth0stanley), Joel Lehman (@joelbot3000), Jeff Clune (@jeffclune), Nick Cheney (@CheneyLab), Richard Song (@XingyouSong), Chelsea Finn (@chelseabfinn), Julian Togelius (@togelius), Sam Earle (@Smearle_RH), Hod Lipson (@hodlipson), and Jean-Baptiste Mouret (@jb_mouret).This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
Headed to #neurips2025? Join the @LilaSciences team at our reception on Fri, Dec 5 to connect with leaders and innovators across AI & ML. I hope to see you there! luma.com/LilaNeurIPS2025
-
Kenneth Stanley reposted
Mizuki Oka/岡瑞起
@miz_oka
目標という幻想。目標と立てると大きな成果が生まれない?
Kenneth Stanley @kenneth0stanley とJoel LehmanのPivot動画配信されました。KPIとOKRに悩まされている方必見です。
youtu.be/YsYLx9Z_FBw -
Kenneth Stanley
@kenneth0stanley
It was a real honor to appear with @joelbot3000 on the Japanese program Pivot while in Japan for release of the Japanese edition of Why Greatness Cannot Be Planned. This is actually an in-person interview with both me and Joel: youtu.be/YsYLx9Z_FBw
-
Kenneth Stanley reposted
hardmaru
@hardmaru
Excited to announce Sakana AI’s Series B! 🐟
From day one, Sakana AI has done things differently. Our research has always focused on developing efficient AI technology sustainably, driven by the belief that resource constraints-not limitless compute-are key to true innovation. This principle extends directly to our business model, which we built with that same focus on sustainability and profitability.
Our company is dedicated to deploying our research into Japan’s key business and public sectors. I’m so proud of the fantastic, growing enterprise AI business we’ve built over the past year in Japan. We have partnered with some of the largest enterprises in Japan, focusing on AI applications that deliver real, practical benefits to our clients. This funding will accelerate our mission: to develop frontier AI sustainably and implement technology that truly benefits Japan.
We are honored to receive support from new and existing investors, including MUFG, Khosla Ventures, Factorial, Macquarie Capital, Fundomo, Mouro Capital, NEA, Geodesic, Lux Capital, Ora Global, MPower Partners, Shikoku Electric, and IQT. -
Kenneth Stanley
@kenneth0stanley
The reason science is not just about compression is that you cannot compress what you have not yet observed, but often the greatest insights are about how the unobserved world might work.
-
Kenneth Stanley reposted
Roberto Dailey
@RobertoDailey1
New work from Cognizant AI lab: Solving a Million-step LLM Task with Zero Errors.
Existing LLMs struggle on long task horizons as persistent error rates compound, even when the LLMs know how to solve the task. Apple’s “Illusion of thinking” demonstrated that state of the art reasoning models could struggle with a simple task, Towers of Hanoi, if that task required execution of hundreds of steps in a row without error.
We hypothesized we could see much higher performance by taking breaking down the task into its smallest subtasks, then using voting and red flagging to boost subtask accuracy.
With these simple modifications we were able to push the simple gpt-4.1-mini to solve the 20-disk towers of Hanoi, or 1,048,575 steps without a single error! Seeing these results we believe with the right, robust, frameworks, LLMs can be scaled to vastly longer task lengths than their base model.
Paper: arxiv.org/abs/2511.09030
Blog: www.cognizant.com/us/en/ai-lab/blog/makerThis media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Thrilled to share what we've been working on! Open-ended self-improvement for many complex skills in complex worlds, including infinitely many Genie worlds. Exciting! 🚀🌍✨
A huge congrats to the wonderful Sima team. 🎉
Google DeepMind
@GoogleDeepMind
SIMA 2 is our most capable AI agent for virtual 3D worlds. 👾🌐
Powered by Gemini, it goes beyond following basic instructions to think, understand, and take actions in interactive environments – meaning you can talk to it through text, voice, or even images. Here’s how 🧵This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
So an AI-generated song hit number 1 on the country charts. But what was the extent of the human input? The view that “AI” will start displacing human artists makes less sense if getting to number 1 still requires a lot of human input. If the human input is the deciding factor, then arguably the human using the AI is just a new kind of artist.
The lead singer in the recording is not human. But if the songwriter is human, is the songwriter not an artist? If a songwriter collaborates with a human singer, there is little controversy over whether the songwriter is an artist.
Then you might ask what does “songwriter” actually mean here. Did a human write just some lyrics? Or did they write lyrics and melody, say by recoding a draft rendition in an audio file? Or maybe they even recorded instrumentals too, and the AI just riffed and remixed it and made it sound good. The details matter for understanding whether there is a human artist behind this work.
If the human contributed minimally, say by simply prompting for a country song about living by one’s own rules, then it would be true that the AI is genuinely showing signs of displacing human artists. But if the human contributed critically and at high granularity, it’s more nuanced, and rather than displacing artists it seems more like creating a new category of artists - artists that previously had no realistic path for expressing their inspirations, even if they had some intrinsic (though locked up) talent.
Let’s suppose indeed that there is this new category opening up. Then you could argue that these “new artists” are inferior to the “real artists” because they needed a crutch to express themselves. But if their creative inspiration (which is indeed still human in its origin) is now bringing joy or wonder when before it could not, maybe it’s fair to say that AI has now minted new kinds of human artists.
Of course we don’t know the extent of the human contribution in Walk My Walk. I think that contribution should be disclosed explicitly (for any art involving AI) so we can understand better what’s happening and whether what I said above even applies. Just as popular music often tells us who wrote the song, who played the instruments, etc., we deserve to know who did what when AI is involved. Then we could understand better what kind of world we’re creating and also decide if we want to be listening to or experiencing specific artifacts with little individual human contribution. -
Kenneth Stanley
@kenneth0stanley
Here @jposhaughnessy got me on camera making a point about education and goals that I think is worth some contemplation. We don’t begin our lives thinking everything we do needs to be driven by a defined objective. The educational system teaches us that!
Infinite Loops 🎙
@InfiniteL88ps
"Sometimes I think about children because I think you don't see it in their firmware. You bring a little toddler to the playground, they're not going to say, 'What are my goals for today?' I'm going to learn how to climb that monkey bar. That's what I like. And then I'll be better at monkey bars next time. They don't think objectively at all. It's just like, they see a playground. It's just endless opportunities."
- @Kenneth0Stanley on how desire for objectives is drilled into us. -
Kenneth Stanley reposted
Blas
@BlasMoros
super excited to share this fun release!
it’s the first building block in our social layer
now you can like and comment on every app, follow your friends to see what apps they’ve published, and even see remix and download counts
can’t wait to see what you think!
wabi
@wabi
what if the app store was social?
with wabi, it is.
introducing the social app store where you can instantly see what apps your friends are creating, loving, and remixing -
Kenneth Stanley reposted
Jim O'Shaughnessy
@jposhaughnessy
“If you don’t pursue things that interest you, you won’t do anything interesting.”
~@kenneth0stanley
Infinite Loops 🎙
@InfiniteL88ps
🚨 New episode 🚨
Ken Stanley (@kenneth0stanley) – AI researcher and co-author of Why Greatness Cannot Be Planned – joins the show to discuss the power of open-endedness, ending the tyranny of objective thinking, the value of pursuing interesting things and our current woes in academia and science funding.
Timestamps:
0:00:00 Intro
0:01:40 Are We Deterministic Thinkers Living in a Probabilistic World?
0:06:43 Evolving Complexity and the NEAT Algorithm
0:13:50 The Origins of Picbreeder
0:22:53 The Power of Novelty Search
0:28:04 The Cultural Impact of Non-Objective Thinking
0:31:31 Why Pursuing Interestingess is Not the Same as Being Random
0:39:07 Mechanizing Serendipity via Stepping Stones
0:47:27 The Allure of the Security Blanket
0:56:51 The Omni-culture of NSF Funding
1:03:05 What Makes an Innovator?
1:06:54 The Fractured Entangled Representation Hypothesis
1:13:40 What Does it Say About LLMs if They Are Fractured and Entangled?
1:20:25 Path Dependence and Careers
1:24:21 Ken as Emperor of the World
👇👇This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
I had a great time talking to @jposhaughnessy on the @InfiniteL88ps podcast. Jim and I share a lot of views in common though coming from different professional backgrounds- probably a sign we're noticing some things that are fundamental!
Infinite Loops 🎙
@InfiniteL88ps
🚨 New episode 🚨
Ken Stanley (@kenneth0stanley) – AI researcher and co-author of Why Greatness Cannot Be Planned – joins the show to discuss the power of open-endedness, ending the tyranny of objective thinking, the value of pursuing interesting things and our current woes in academia and science funding.
Timestamps:
0:00:00 Intro
0:01:40 Are We Deterministic Thinkers Living in a Probabilistic World?
0:06:43 Evolving Complexity and the NEAT Algorithm
0:13:50 The Origins of Picbreeder
0:22:53 The Power of Novelty Search
0:28:04 The Cultural Impact of Non-Objective Thinking
0:31:31 Why Pursuing Interestingess is Not the Same as Being Random
0:39:07 Mechanizing Serendipity via Stepping Stones
0:47:27 The Allure of the Security Blanket
0:56:51 The Omni-culture of NSF Funding
1:03:05 What Makes an Innovator?
1:06:54 The Fractured Entangled Representation Hypothesis
1:13:40 What Does it Say About LLMs if They Are Fractured and Entangled?
1:20:25 Path Dependence and Careers
1:24:21 Ken as Emperor of the World
👇👇This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
Performing differently in response to the same questions depending on in which language they’re posed is another sign pointing to the presence of fractured entangled representation (FER) in LLMs.
Lossfunk
@lossfunk
We dropped our explorations on how language impacts LLM reasoning in a new blog post!
This work - done by our research intern @Madbonze16 - has many insights.
1st insight 👉 If the same math question is asked in Telugu, performance drops! -
Kenneth Stanley
@kenneth0stanley
The way we talk about “slop” is an interesting cultural signal. It’s a kind of new lens for thinking about what makes something meaningful. Two questions for you: (1) In your opinion, is all AI-generated content slop?
If not, (2) what kind of AI-generated content is *not* slop? -
Kenneth Stanley
@kenneth0stanley
This baking technology from @ai_bread is worth checking out. Models should be able to update their weights based on prompting alone, and baking does that. The implications for ongoing change of internal representation based on experience is particularly interesting.
Bread
@ai_bread
Announcing Bread Technologies. We’re building machines that learn like humans.
We raised a $5 million seed round led by Menlo Ventures and have been building in stealth for 10 months.
Today, we rise 🍞This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Kai Arulkumaran
@kaixhin
Today we're celebrating the Japanese book launch of "Why Greatness Cannot Be Planned" by @kenneth0stanley @joelbot3000 🧬
-
Kenneth Stanley reposted
Andrew Beam
@AndrewLBeam
Today we’re excited to share additional support in the Series A for Lila: $350M in total for the round and $550M raised to date. I’m grateful for our team, our early partners, and the investors who believe in this mission.
In an earlier post I asked whether science can create the next scaling paradigm for AI. We believe the way forward is to run the scientific method at scale—hypothesize → design → run → measure → update—and let models learn from real experiments.
This financing helps us do that. We’re building the world’s largest verifier for science to train a new kind of AI model.
Thank you to the partners from our first close and to the new investors who joined alongside them.
If you’re excited about creating a model that learns from the scientific method itself—or you’re a researcher or engineer who wants to work on important problems—we’d love to talk.
Lila Sciences
@LilaSciences
We’ve closed our Series A. With $550M total raised since launch, we’re ready to scale.
🚀Learn how our Series A is fueling our next chapter: www.lila.ai/news/announcing-the-close-of-our-series-a -
Kenneth Stanley
@kenneth0stanley
Turns out our Series A grew larger. All the more reason @LilaSciences is an exciting new frontier AI lab, and one of the top places in the world to work on on open-ended discovery.
Lila Sciences
@LilaSciences
We’ve closed our Series A. With $550M total raised since launch, we’re ready to scale.
🚀Learn how our Series A is fueling our next chapter: www.lila.ai/news/announcing-the-close-of-our-series-a -
Kenneth Stanley reposted
Shreyas Pulle
@shreyaspulle
As someone very early in their AI learning journey, reading @kenneth0stanley and @joelbot3000’s “Why Greatness Cannot Be Planned” has been a truly foundational philosophical text. Would highly recommend to those who live their life thinking a clear objective function is needed to “suceed”. I was this person (and maybe still am). “Shoot for the stars, reach the moon” is something I have consistently told myself. However, the book is allowing me to shift my paradigm to one of following “interestingness”, and in reality that often means following “uncomfortableness”.
Picking up the book, I knew it would shake the foundation of the way I live my life, but it has been the most interesting philosophical stepping stone.
It is fascinating to think of the applications from AI (how to imbue creativity into the LLMs we all use), to material science (finding truly novel materials like a room-temperature superductor) or entrepreneurship. The application in entrepreneurship are especially interesting. @JeffBezos wrote in early shareholder letters that you cannot run a truly successful business solving for the short-term profit/efficiency gains. Instead a business should be guided by core principles (for Amazon it was customer obsession, differentiation and constantly taking risky bets) This, in turn, guides the business to a novel area of the corporation search space. It is in that search space where the treasure lies. That treasure is what people most commonly term “the moat”.
My main guiding principle is drawing upon @naval’s advice for everyone to “follow your inspiration, because inspiration is fleeting” (paraphrasing). Following @kenneth0stanley and @joelbot3000’s book, I am starting to see life as a process of collecting the most inspiration-driven and interesting stepping stones. I trust that whether or not greatness is achieved, it'll be a lot of fun in the process. -
Kenneth Stanley reposted
Kai Arulkumaran
@kaixhin
Congrats to @akarshkumar0101 for winning the best paper talk award at #alife2025! 👏
-
Kenneth Stanley reposted
Mizuki Oka/岡瑞起
@miz_oka
🎉 Book Launch Event: "Why Greatness Cannot Be Planned" (目標という幻想)
Join authors Kenneth O. Stanley @kenneth0stanley & Joel Lehman in Tokyo! 📅 Oct 18 (Sat) 14:00-16:00 📍 Design Hub, Roppongi 🎤 Moderated by Madoka Sawa @madoka510 (former Microsoft)
Exploring AI, ALife & the future beyond objectives.
Free entry | Limited to 70 Register: alife-bnn202510.peatix.com
#alife #ai -
Kenneth Stanley reposted
Sara Hooker
@sarahookr
I'm starting a new project.
Working on what I consider to be the most important problem: building thinking machines that adapt and continuously learn.
We have incredibly talent dense founding team + are hiring for engineering, ops, design.
Join us: adaptionlabs.ai -
Kenneth Stanley
@kenneth0stanley
Nice to see an exploration of the potential for ES (evolution strategies) in LLM fine tuning! Many potential advantages are discussed in this thread from @yule_gan .
Yulu Gan
@yule_gan
Reinforcement Learning (RL) has long been the dominant method for fine-tuning, powering many state-of-the-art LLMs. Methods like PPO and GRPO explore in action space. But can we instead explore directly in parameter space? YES we can. We propose a scalable framework for full-parameter fine-tuning using Evolution Strategies (ES).
By skipping gradients and optimizing directly in parameter space, ES achieves more accurate, efficient, and stable fine-tuning.
Paper: arxiv.org/pdf/2509.24372
Code: github.com/VsonicV/es-fine-tuning-paper -
Kenneth Stanley
@kenneth0stanley
Image and video models are a great place to find examples of fractured entangled representation (FER) like this one in Sora 2. The visual output format lends itself to dramatic examples. Here we see that the rules of physics are entangled with color!
There are likely innumerable such examples to be found. Seeing them is helpful for understanding the way the underlying representation organizes knowledge, and ultimately towards pointing us to ways to mitigate the issue.
Credit to @kparikh2001 for highlighting this post and connecting it to FER initially in his own thread.
Chase Brower
@ChaseBrowe32432
i was expecting that it would be indeed overfit, but that it would take more than just changing the colors to break it... maybe not?
-
Kenneth Stanley reposted
Ciaran
@ciaran_regan_
Why Greatness Greatness Cannot Be Planned authors @kenneth0stanley and @joelbot3000 will be giving a talk on October 18th in Roppongi! I'm planning to go!
-
Kenneth Stanley reposted
Michael Chang
@mmmbchang
This is a new kind of social media - one that incentivizes creation and sharing, not simply just consumption.
Social is a crucial component to creativity: seeing others' creations inspires your own.
Eventually I expect we will the emergence of something like Github for media creation, where people can have version control, build on others' work, and iterate on art masterpieces they are proud of.
Remixing is the core feature, native to Sora, that makes this social experience possible. The Sora team really cooked on making it super easy to riff on other's ideas! Truly a "Greatness Cannot Be Planned" (h/t @kenneth0stanley @joelbot3000 ) phenomenon, but at global scale.
Here is a fun video I made of @ericmintun riding a wiener dog 😀 and a remix I made by giving the dog a pink coat with polka dots and a chicken head (h/t @RubenEVillegas )
OpenAI
@OpenAI
This is the Sora app, powered by Sora 2.
Inside the app, you can create, remix, and bring yourself or your friends into the scene through cameos—all within a customizable feed designed just for Sora videos.
See inside the Sora app👇This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
Congrats to @kanjun and @imbue_ai on the release of Sculptor. One thing I know is that they found this very interesting stepping stone by following the ethos of Why Greatness Cannot Be Planned, and it will be exciting to see where it leads!
Kanjun 🐙
@kanjun
Sculptor: the missing UI for Claude Code 🎨
Imagine running 5 Claudes in parallel, safely in containers, while you stay in flow. Then bring their work straight into your IDE to test/edit together.
This is how one developer ships like a team. Try it with Sonnet 4.5!This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
So much controversy triggered by claims about how humans learn. But the deeper question is why the way we learn works. We need to understand the why to know in what way “how” matters. Nature is inspiration for AI, not prescription. The crux of the whole debate is abstraction.
Underneath all learning is representation. The key question is not whether there is a “world model” or whether there is imitation or not. The instrumental question is, what kind of representation does the learning mechanism produce? And then, what are all the leaning mechanisms, natural or not, that can produce good representations?
Dwarkesh Patel
@dwarkesh_sp
.@RichardSSutton, father of reinforcement learning, doesn’t think LLMs are bitter-lesson-pilled.
My steel man of Richard’s position: we need some new architecture to enable continual (on-the-job) learning.
And if we have continual learning, we don't need a special training phase - the agent just learns on-the-fly - like all humans, and indeed, like all animals.
This new paradigm will render our current approach with LLMs obsolete.
I did my best to represent the view that LLMs will function as the foundation on which this experiential learning can happen. Some sparks flew.
0:00:00 – Are LLMs a dead-end?
0:13:51 – Do humans do imitation learning?
0:23:57 – The Era of Experience
0:34:25 – Current architectures generalize poorly out of distribution
0:42:17 – Surprises in the AI field
0:47:28 – Will The Bitter Lesson still apply after AGI?
0:54:35 – Succession to AIThis media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Sakana AI
@SakanaAILabs
We’re excited to introduce ShinkaEvolve: An open-source framework that evolves programs for scientific discovery with unprecedented sample-efficiency.
Blog: sakana.ai/shinka-evolve/
Code: github.com/SakanaAI/ShinkaEvolve
Like AlphaEvolve and its variants, our framework leverages LLMs to find state-of-the-art solutions to complex problems, but using orders of magnitude fewer resources!
Many evolutionary AI systems are powerful but act like brute-force engines, burning thousands of samples to find good solutions. This makes discovery slow and expensive. We took inspiration from the efficiency of nature. ‘Shinka’ (進化) is Japanese for evolution, and we designed our system to be just as resourceful.
On the classic circle packing optimization problem, ShinkaEvolve discovered a new state-of-the-art solution using only 150 samples. This is a big leap in efficiency compared to previous methods that required thousands of evaluations.
We applied ShinkaEvolve to a diverse set of hard problems with real-world applications:
1/ AIME Math Reasoning: It evolved sophisticated agentic scaffolds that significantly outperform strong baselines, discovering an entire Pareto frontier of solutions trading performance for efficiency.
2/ Competitive Programming: On ALE-Bench (a benchmark for NP-Hard optimization problems), ShinkaEvolve took the best existing agent's solutions and improved them, turning a 5th place solution on one task into a 2nd place leaderboard rank in a competitive programming competition.
3/ LLM Training: We even turned ShinkaEvolve inward to improve LLMs themselves. It tackled the open challenge of designing load balancing losses for Mixture-of-Experts (MoE) models. It discovered a novel loss function that leads to better expert specialization and consistently improves model performance and perplexity.
ShinkaEvolve achieves its remarkable sample-efficiency through three key innovations that work together: (1) an adaptive parent sampling strategy to balance exploration and exploitation, (2) novelty-based rejection filtering to avoid redundant work, and (3) a bandit-based LLM ensemble that dynamically picks the best model for the job.
By making ShinkaEvolve open-source and highly sample-efficient, our goal is to democratize access to advanced, open-ended discovery tools. Our vision for ShinkaEvolve is to be an easy-to-use companion tool to help scientists and engineers with their daily work. We believe that building more efficient, nature-inspired systems is key to unlocking the future of AI-driven scientific research. We are excited to see what the community builds with it!
Learn more in our technical report: arxiv.org/abs/2509.19349 -
Kenneth Stanley reposted
Sakana AI
@SakanaAILabs
“Automating the Search for Artificial Life With Foundation Models” is now published in the Artificial Life Journal! 🦎🧠
Article: direct.mit.edu/artl/article/31/3/368/132866/Automating-th…
ASAL is a method using foundation models to automate the discovery of new artificial lifeforms, accelerating ALIFE research. -
Kenneth Stanley reposted
Foresight Institute
@foresightinst
Upcoming online seminar tomorrow with @drmichaellevin, @tarinziyaee and @DoctorJosh!
Register: luma.com/ndpskbkn -
Kenneth Stanley reposted
Andrew Beam
@AndrewLBeam
Can science create the next scaling paradigm for AI?
We think so, and we'll be using our recent Series A raise to build a new kind of scientific reasoning model at scale.
If you're interested in how we think all of science is subject to the bitter lesson, read on in the 🧵👇
Lila Sciences
@LilaSciences
🚀 Big milestone: We raised a $235M Series A, co-led by Braidwell and Collective Global! Our team is excited for this next chapter in building the world’s first scientific superintelligence platform to accelerate breakthroughs in life, chemical, and materials sciences.
📣 If the idea of building scientific superintelligence excites you, come build with us. We’re seeking extraordinary, driven people across AI research, software, robotics, biology, chemistry, materials science, and operations. www.lila.ai/open-roles
#scientificsuperintelligence #ai #innovation -
Kenneth Stanley
@kenneth0stanley
A big day for @LilaSciences , where I’m leading open-endedness. A frontier lab focused exclusively on advancing the singularly open-ended enterprise of science can make bold and unique bets that are off the radar of the chatbot giants. It’s a great place to work and do research!
Lila Sciences
@LilaSciences
🚀 Big milestone: We raised a $235M Series A, co-led by Braidwell and Collective Global! Our team is excited for this next chapter in building the world’s first scientific superintelligence platform to accelerate breakthroughs in life, chemical, and materials sciences.
📣 If the idea of building scientific superintelligence excites you, come build with us. We’re seeking extraordinary, driven people across AI research, software, robotics, biology, chemistry, materials science, and operations. www.lila.ai/open-roles
#scientificsuperintelligence #ai #innovation
Kenneth Stanley
@kenneth0stanley
Open roles (including in open-endedness!): www.lila.ai/open-roles
-
Kenneth Stanley reposted
Lila Sciences
@LilaSciences
🚀 Big milestone: We raised a $235M Series A, co-led by Braidwell and Collective Global! Our team is excited for this next chapter in building the world’s first scientific superintelligence platform to accelerate breakthroughs in life, chemical, and materials sciences.
📣 If the idea of building scientific superintelligence excites you, come build with us. We’re seeking extraordinary, driven people across AI research, software, robotics, biology, chemistry, materials science, and operations. www.lila.ai/open-roles
#scientificsuperintelligence #ai #innovation -
Kenneth Stanley
@kenneth0stanley
Interesting to hear it proudly stated that the most valuable company on Earth “doesn’t have a long-term plan.” Almost a perfect mascot for Why Greatness Cannot Be Planned.
Reads with Ravi
@readswithravi
Jensen Huang on why he don’t wear a watch:
-
Kenneth Stanley reposted
Kradle
@kradleai
Hello World, we’re kradle.ai
We eval frontier models by putting them in simulations.
So what happens when 6 frontier models compete in #minecraft for GPUs?
Video and 🧵This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
ALIFE Conference 2027
@ALifeConf
🔔 SPEAKER ANNOUNCEMENT 🔔
Kenneth O. Stanley will be part of our keynote panel at #alife2025.
X profile: @kenneth0stanley -
Kenneth Stanley reposted
AI Deeply
@AiDeeply
Good pic. Human understanding is clearly more like the image on the left: sparse capture of meaningful factors.
I missed the paper when it came out. It's … genius.
e.g. "The danger of outward performance—of benchmarks—is that they miss this critical distinction. Somewhere beneath the surface, where representations form and churn, there is an explanation for why simply doing well on a test or benchmark does not foretell later creative feats of great distinction. FER may be a critical yet heretofore unappreciated part of that story. Moreover, the hope that FER will vanish with scale—representational optimism—may be misplaced and merits scrutiny and additional research." -
Kenneth Stanley reposted
Machine Learning Street Talk
@MLStreetTalk
A reminder to read this paper from Akarsh, Kenneth, Joel and Jeff. Just like WGCBP caused a revolution in my thinking on AI, this new perspective is the best intuition pump there has ever been to understand what is missing from current AI. When you see it, you can't unsee it - it's that good. AIs which actually understand should carve the world up "by the joints" and form *hard constraints* about about what can happen, and what can be creatively evolved.
Kenneth Stanley
@kenneth0stanley
If you’re struggling to understand what people mean when they say things like “truly understand,” it boils down to Unified Factored Representation (UFR). That’s the foundation behind the slippery intuition.
François Chollet
@fchollet
A student who truly understands F=ma can solve more novel problems than a Transformer that has memorized every physics textbook ever written.
-
Kenneth Stanley
@kenneth0stanley
If you’re struggling to understand what people mean when they say things like “truly understand,” it boils down to Unified Factored Representation (UFR). That’s the foundation behind the slippery intuition.
François Chollet
@fchollet
A student who truly understands F=ma can solve more novel problems than a Transformer that has memorized every physics textbook ever written.
-
Kenneth Stanley reposted
Andy Xu
@4ndyXu
Inspired by @rao2z talk, it's interesting how LLMs are only possible because of the internet, but no one would have thought "I should go create the web to build agi" 40 yrs ago. Another great example of @kenneth0stanley thesis where stepping stones of progress are not intuitive
Subbarao Kambhampati (కంభంపాటి సుబ్బారావు)
@rao2z
📢Slides for the @harveymudd Colloquium are at www.dropbox.com/scl/fi/b54sp4i331e6nwdsdt0a1/hmc-colloq.p…
-
Kenneth Stanley reposted
Rick
@rickasaurus
This book articulates something I’ve fought for my whole career: goals and team metrics lead to mediocre outcomes. Actually novel work must be exploratory.
-
Kenneth Stanley reposted
Reads with Ravi
@readswithravi
This book is a mind-blowing read ‼️
-
Kenneth Stanley
@kenneth0stanley
Carl Jung made a point long ago that both foreshadows fractured entangled representation (FER) and offers a thought-provoking critique of modern ML in general: "Beware of unearned wisdom." (I'd update it to "unearned knowledge" for AI today.)
If the way that you acquire knowledge impacts your facility for applying that knowledge in the future through its consequent underlying representation, then what price do you pay for the unnatural vacuuming up of vast swaths of knowledge in a giant disorganized batch?
Unearned knowledge has a cost that's rarely if ever discussed in AI or ML.
Thank you to @jakobmrees, an undergrad at NYU, for perceptively bringing this quote to my attention! -
Kenneth Stanley reposted
Chris Barber
@chrisbarber
I asked @kenneth0stanley, @finbarrtimbers, and @JeffDean: "What might help society adapt to future AI progress?"
@kenneth0stanley - Lila Sciences, ex-OpenAI, Greatness Cannot Be Planned
"Even though we can't predict exactly what's coming, there's a real need for new institutions to help facilitate this transition."
@finbarrtimbers - Ai2, ex-Midjourney, ex-DeepMind
"There's not that many problems that are pure software problems. A lot of problems are human problems."
@JeffDean - Google DeepMind, Google Research
"I think things that help people identify new opportunities and what kind of skills are needed are pretty important." -
Kenneth Stanley
@kenneth0stanley
A great honor, I’m humbled!
hardmaru
@hardmaru
Why Greatness Cannot Be Planned
Both English and Japanese editions now found a home @SakanaAILabs library ✨ -
Kenneth Stanley
@kenneth0stanley
It’s a physical, real-time Picbreeder!
Cédric
@cedcolas
Just setting up my art exhibition for the week in MIT's Stata center (4th floor, 32-G409)
Come play with a visual modular synthesizer built from a physical compositional neural net
Plug cables, twist knobs and shape amazing patterns in real time!! -
Kenneth Stanley reposted
Sakana AI
@SakanaAILabs
What if we could evolve AI models like organisms in nature, letting them compete, mate, and combine their strengths to produce ever-fitter offspring?
Excited to share our new work: “Competition and Attraction Improve Model Fusion” presented at GECCO’25🦎 where it was a runner-up for best paper!
Paper: arxiv.org/abs/2508.16204
Code: github.com/SakanaAI/natural_niches
Summary of Paper
At Sakana AI, we draw inspiration from nature’s evolutionary processes to build the foundation of future AI systems. Nature doesn’t create one single, monolithic organism; it fosters a diverse ecosystem of specialized individuals that compete, cooperate, and combine their traits to adapt and thrive. We believe AI development can follow a similar path.
What if instead of building one giant monolithic AI, we could evolve a whole ecosystem of specialized models that collaborate and combine their skills? Like a school of fish 🐟, where collective intelligence emerges from the group.
This new paper builds on our previous research on model merging, which follows such an evolutionary path. We started by using evolution to find the best “recipes” to merge existing models (our Nature Machine Intelligence paper: www.nature.com/articles/s42256-024-00975-8). Then, we explored how to maintain diversity to acquire new skills in LLMs (our ICLR 2025 paper: openreview.net/forum?id=Kvdh12wGC0). Now, we're combining these ideas into a full evolutionary system.
A key limitation remained in earlier work: model merging required manually defining how models should be partitioned (e.g., by fixed layer or blocks) before they could be combined. What if we could let evolution figure that out too?
Our new paper proposes M2N2 (Model Merging of Natural Niches), a more fluid method, which overcomes this with three key, nature-inspired ideas:
1/ Evolving Merging Boundaries 🌿: Instead of merging models using pre-defined, static boundaries (e.g. fixed layers), M2N2 dynamically evolves the “split-points” for merging. This allows for a far more flexible and powerful exploration of parameter combinations, like swapping variable-length segments of DNA rather than entire chromosomes.
2/ Diversity through Competition 🐠: To ensure we have a rich pool of models to merge, M2N2 makes them compete for limited resources (i.e., data points in a training set). This forces models to specialize and find their own “niche,” creating a population of diverse, high-performing specialists that are perfect for merging.
3/ Attraction and Mate Selection 💏: Merging models can be computationally expensive. M2N2 introduces an “attraction” heuristic that intelligently pairs models for fusion based on their complementary strengths—choosing partners that perform well where the other is weak. This makes the evolutionary search much more efficient.
Does it work?
The results are fascinating: This is the first time model merging has been used to evolve models entirely from scratch, outperforming other evolutionary algorithms. In one experiment, starting with random networks, M2N2 evolved an MNIST classifier that achieves performance comparable to CMA-ES, but is far more computationally efficient.
Does it scale?
We also showed that M2N2 can scale to large, pre-trained models: We used M2N2 to merge a math specialist LLM with an agentic specialist LLM. M2N2 produced a merged model that excelled at both math and web shopping tasks, significantly outperforming other methods. The flexible split-point was crucial here.
Does it work on multimodal models?
When we applied M2N2 to text-to-image models, we merged several models by adapting them only for Japanese prompts. The resulting model not only improved on Japanese but also retained its strong English capabilities—a key advantage over fine-tuning, which can suffer from catastrophic forgetting.
This nature-inspired approach is central to Sakana AI’s mission to find new foundations for AI based on collective intelligence. Rather than scaling monolithic models, we envision a future where ecosystems of diverse, specialized models co-evolve, collaborate, and combine, leading to more adaptive, robust, and creative AI. 🐙
We hope this work sparks more interest in these under-explored ideas!
Published in ACM GECCO’25: Proceedings of the Genetic and Evolutionary Computation Conference. DOI: doi.org/10.1145/3712256.3726329 -
Kenneth Stanley reposted
helena sarin
@NeuralBricolage
perhaps that really turned into my artistic goal - rescuing weirdness from the flood of optimization - one potteryGAN artifact at a time - to riff on @kenneth0stanley book idea - the weirdness can't be planned - AI of yesteryear was unintentionally weird and funny - you can't parametrize this
helena sarin
@NeuralBricolage
"Keep AI Weird" Campaign,
the strategic planning committee -
Kenneth Stanley reposted
Julian Togelius
@togelius
Can large language models play simple arcade games? Kind of. Sometimes. Slowly, and not as well as a simple search algorithm. And only if you format the input right. Of course, we made a benchmark to investigate this in more detail, because that's what we do.
-
Kenneth Stanley
@kenneth0stanley
Sad news. Margaret Boden’s book The Creative Mind influenced my thinking on creativity early on. Her notion of transformational creativity remains relevant today. Of all the things people knock LLMs for lacking, transformational creativity is the true prize.
nature
@Nature
Obituary: Margaret Boden (1936-2025) cognitive scientist who explored how machines might emulate human imagination
go.nature.com/4fzsdcu -
Kenneth Stanley reposted
nature
@Nature
Obituary: Margaret Boden (1936-2025) cognitive scientist who explored how machines might emulate human imagination
go.nature.com/4fzsdcu -
Kenneth Stanley reposted
Bryon Tjanaka
@btjanaka
Excited to share our new @pyribs tutorial on Quality Diversity through AI Feedback (QDAIF)! This tutorial integrates pyribs with LLMs (via @langchain & @ollama) to write diverse stories about "a suspicious spy and a rich politician." Available here: docs.pyribs.org/en/stable/tutorials/qdaif.html
-
Kenneth Stanley reposted
camenduru
@camenduru
I started studying Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis by @akarshkumar0101, @jeffclune, @joelbot3000, and @kenneth0stanley.
📄paper: arxiv.org/abs/2505.11581
Thanks to @MrForExample ❤ for sharing it.
What's your opinion on the paper? -
Kenneth Stanley
@kenneth0stanley
.@jim_rutt took me on a real deep dive into the Fractured Entangled Representation (FER) Hypothesis on his podcast. Worth a listen if you’re looking for more on this topic:
-
Kenneth Stanley reposted
Lili
@lchen915
Self-Questioning Language Models: LLMs that learn to generate their own questions and answers via asymmetric self-play RL.
There is no external training data – the only input is a single prompt specifying the topic. -
Kenneth Stanley
@kenneth0stanley
benchmarkmaxxing = the objective paradox = the myth of the objective = Why Greatness Cannot Be Planned = the security blanket of objectives = what’s happening now is a predictable outcome of objective obsession.
Andrej Karpathy
@karpathy
I'm noticing that due to (I think?) a lot of benchmarkmaxxing on long horizon tasks, LLMs are becoming a little too agentic by default, a little beyond my average use case.
For example in coding, the models now tend to reason for a fairly long time, they have an inclination to start listing and grepping files all across the entire repo, they do repeated web searchers, they over-analyze and over-think little rare edge cases even in code that is knowingly incomplete and under active development, and often come back ~minutes later even for simple queries.
This might make sense for long-running tasks but it's less of a good fit for more "in the loop" iterated development that I still do a lot of, or if I'm just looking for a quick spot check before running a script, just in case I got some indexing wrong or made some dumb error. So I find myself quite often stopping the LLMs with variations of "Stop, you're way overthinking this. Look at only this single file. Do not use any tools. Do not over-engineer", etc.
Basically as the default starts to slowly creep into the "ultrathink" super agentic mode, I feel a need for the reverse, and more generally good ways to indicate or communicate intent / stakes, from "just have a quick look" all the way to "go off for 30 minutes, come back when absolutely certain". -
Kenneth Stanley reposted
TechnicalBen
@_TechyBen
@VictorTaelin Sometimes.
Sometimes I wish my algorithm also gave me the lottery numbers.
But for now, I will be content with the gems like this.
Amazing.
(In one image, the clearer description of the complexities and solutions than I've ever seen)
Kenneth Stanley
@kenneth0stanley
Those who intuit something is lacking in LLMs struggle to pinpoint the gap beyond inadequate metaphors like “stochastic parrot” or “glorified autocomplete.” What you’re groping for is fractured entangled representation (FER). That’s the concrete crux of your slippery intuitions.
It’s why these models can keep improving in dramatic and genuinely nontrivial ways yet still suffer from an underlying sickness. It’s where real opportunity and hope lie for revolutionary progress,
Representation is at the heart of thought, of creativity, and indeed of intelligence itself. It is possible to “know” an unfathomable amount and yet still represent it poorly.
And it’s possible for representation to be better. And it will ultimately happen one way or another. -
Kenneth Stanley reposted
Luis Lamb
@luislamb
Valuable insights from @kenneth0stanley on LLMs
Kenneth Stanley
@kenneth0stanley
Those who intuit something is lacking in LLMs struggle to pinpoint the gap beyond inadequate metaphors like “stochastic parrot” or “glorified autocomplete.” What you’re groping for is fractured entangled representation (FER). That’s the concrete crux of your slippery intuitions.
It’s why these models can keep improving in dramatic and genuinely nontrivial ways yet still suffer from an underlying sickness. It’s where real opportunity and hope lie for revolutionary progress,
Representation is at the heart of thought, of creativity, and indeed of intelligence itself. It is possible to “know” an unfathomable amount and yet still represent it poorly.
And it’s possible for representation to be better. And it will ultimately happen one way or another. -
Kenneth Stanley reposted
James Chen
@jchencxh
A type of the best work unify tip-of-the-tongue thoughts everyone has into one coherent mental model.
FER is one of them.
Kenneth Stanley
@kenneth0stanley
Those who intuit something is lacking in LLMs struggle to pinpoint the gap beyond inadequate metaphors like “stochastic parrot” or “glorified autocomplete.” What you’re groping for is fractured entangled representation (FER). That’s the concrete crux of your slippery intuitions.
It’s why these models can keep improving in dramatic and genuinely nontrivial ways yet still suffer from an underlying sickness. It’s where real opportunity and hope lie for revolutionary progress,
Representation is at the heart of thought, of creativity, and indeed of intelligence itself. It is possible to “know” an unfathomable amount and yet still represent it poorly.
And it’s possible for representation to be better. And it will ultimately happen one way or another. -
Kenneth Stanley
@kenneth0stanley
Those who intuit something is lacking in LLMs struggle to pinpoint the gap beyond inadequate metaphors like “stochastic parrot” or “glorified autocomplete.” What you’re groping for is fractured entangled representation (FER). That’s the concrete crux of your slippery intuitions.
It’s why these models can keep improving in dramatic and genuinely nontrivial ways yet still suffer from an underlying sickness. It’s where real opportunity and hope lie for revolutionary progress,
Representation is at the heart of thought, of creativity, and indeed of intelligence itself. It is possible to “know” an unfathomable amount and yet still represent it poorly.
And it’s possible for representation to be better. And it will ultimately happen one way or another. -
Kenneth Stanley
@kenneth0stanley
An interesting stepping stone opens up a new playground. A new playground leads to serendipity. Genie 3 is definitely a new playground.
Tim Rocktäschel
@_rockt
This was such a mind-blowing example of open-ended serendipitous discovery. @jkbr_ai was experimenting with video prompting of Genie 3. One day he accidentally prompted it with a conflicting text world description and Genie 3 just mashed up the two worlds perfectly 🤯
Jakob Bauer
@jkbr_ai
Something we discovered by accident: what happens if we start Genie 3 from a video and a completely unrelated prompt? Turns out the model really, really wants to make it work, to the point where it emulates itself.
The prompt in this one is about a trex on a tropical island. -
Kenneth Stanley reposted
Tim Rocktäschel
@_rockt
This was such a mind-blowing example of open-ended serendipitous discovery. @jkbr_ai was experimenting with video prompting of Genie 3. One day he accidentally prompted it with a conflicting text world description and Genie 3 just mashed up the two worlds perfectly 🤯
Jakob Bauer
@jkbr_ai
Something we discovered by accident: what happens if we start Genie 3 from a video and a completely unrelated prompt? Turns out the model really, really wants to make it work, to the point where it emulates itself.
The prompt in this one is about a trex on a tropical island. -
Kenneth Stanley reposted
Tim Rocktäschel
@_rockt
Harder, Better, Faster, Stronger, Real-time! We are excited to reveal Genie 3, our most capable real-time foundational world model. Fantastic cross-team effort led by @jparkerholder and @shlomifruchter. Below some interactive worlds and capabilities that were highlights for me 🌎👇
This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
dave payne
@davempayne
Five books helped shape my thinking as I was getting @goodstuffnearby off the ground.
Interesting...only one of them is a "business book." -
Kenneth Stanley reposted
Mizuki Oka/岡瑞起
@miz_oka
元OpenAI研究者でOpen-endedness研究の第一人者、ケン・スタンリー@kenneth0stanley とジョエル・リーマン@joelbot3000 による著書です!
『目標という幻想:未知なる成果をもたらす、〈オープンエンド〉なアプローチ』(BNN)
OpenAIのサム・アルトマンが「研究管理の考え方を根本から変えた」と語る一冊でもあります。
"私がOpenAIに入ったときには、普通のソフトウェア企業と同じように四半期ごとの目標を設定すればいいと考えていました...ケン・スタンリーの『偉大なことは計画できない(Greatness Cannot Be Planned)』講演を聞いて、研究マネジメントへの考え方が根本から変わりました"
x.com/kenneth0stanley/status/1677018971579777024
監修:岡瑞起、翻訳:牧尾晴喜
解説:岡瑞起・鈴木健@kensuzuki
Sakana AI
@SakanaAILabs
Kenneth Stanley & Joel Lehmanによる名著『Why Greatness Cannot Be Planned』の日本語版がBNN社より刊行されました!
『目標という幻想:未知なる成果をもたらす、〈オープンエンド〉なアプローチ』
監修:岡瑞起、翻訳:牧尾晴喜、解説:岡瑞起・鈴木健
本書は、科学・技術・芸術・ビジネスなど 、あらゆる領域でブレークスルーを起こすための「目標を定めない」オープンエンドなアプローチを提唱しています。
「ほぼすべての重要な発明の前提条件は、その重要な発明を意図して作られたわけではない」(p.111) ━━この一文が示すように、これは技術開発の本質を突く、極めて重要な考え方です。
私たちSakana AIは、この本に深く触発され、創業当初から研究開発とビジネスの両面でオープンエンドなアプローチを実践してきました。最近では、Google DeepMindのようなフロンティアラボもこの考え方を取り入れ始めており、Tim Rocktäschel氏(@_rockt)らが率いるOpen-Endedness Teamも存在します。
このアプローチは、これまで進化的アルゴリズムや人工生命といった分野の研究者に強く支持されてきました。日本では、本書の邦訳にご尽力された岡瑞起さん(@miz_oka)や鈴木健さん(@kensuzuki)ら、一部の研究者によって伝えられてきました。しかし、多くの巨大テック企業では、依然として目標駆動型のアプローチが主流です。
私たちは、それらとは一線を画す本書のオープンエンドな思考こそが、真のイノベーションを切り拓くと確信しています。日本語版解説で強調されている「複数の価値観や知性が共存し、連携しながら進化する『Plurality(多元性)』」という観点は、より多元的なAI開発を導くものでもあります。
本書の刊行を機に、日本でオープンエンドネスの考え方がさらに広まることを願います。翻訳に関わられた皆様に、心より感謝申し上げます。 -
Kenneth Stanley
@kenneth0stanley
This insightful thread from @MLStreetTalk has triggered interesting discussion on the nature of creativity. I’d add that we should also pull representation into the discussion: what you can imagine is a function of how you represent the world. It’s through representation that you can respect deeply (or violate in an interesting way) the phylogeny of thought that precedes you. An interesting corollary: you can develop a penetrating representation even without a high level of what people conventionally call “intelligence,” which aligns with the point @MLStreetTalk makes about intelligence versus creativity here too.
Machine Learning Street Talk
@MLStreetTalk
I've had a lot of time to think about what creativity is.
I think I finally grok it.
Its much more of a "knowledge thing" than an "intelligence thing".
Creativity is correlated with intelligence, because intelligent people understand things deeper and faster (intelligence is the efficiency of knowledge acquisition). But I was mistaken before to think of creativity as being the other side of the coin of intelligence. It's possible to be creative with deep understanding and almost zero intelligence. Creativity is quite domain-dependent, I couldn't choreograph a fight scene but could write creatively about AI.
So here it is:
"Creativity is the ability to take the next step in an epistemic lineage or phylogeny which respects the constraints of the ancestry and is maximally *evolvable* going forwards in time."
The "inventiveness" or "novelty" of a step is something like; "how tenuous that step is" i.e. is it stepping out into a radically new domain, or is it merging together different epistemic phylogenies through analogy.
The reason why LLMs are not creative is simply that they do not understand anything deeply and can not therefore, even in principle, take logical steps while respecting ancestral constraints (sans direct human supervision). An LLM is like the artist who draws faces with tracing paper, without any deep understanding.
We should also be careful to separate out intrinsic and social creativity. Solving math (or anything verifiable) is one thing, but finding a good (creative) youtube thumb or trading strategy is a constructive and interactive process, it's not something you can do "offline".
I would love to take credit for this, but @kenneth0stanley intuitively knew this all along, perhaps I have just put it more clearly after much reflection (also @akarshkumar0101 was very helpful in analysing this) -
Kenneth Stanley reposted
Machine Learning Street Talk
@MLStreetTalk
I've had a lot of time to think about what creativity is.
I think I finally grok it.
Its much more of a "knowledge thing" than an "intelligence thing".
Creativity is correlated with intelligence, because intelligent people understand things deeper and faster (intelligence is the efficiency of knowledge acquisition). But I was mistaken before to think of creativity as being the other side of the coin of intelligence. It's possible to be creative with deep understanding and almost zero intelligence. Creativity is quite domain-dependent, I couldn't choreograph a fight scene but could write creatively about AI.
So here it is:
"Creativity is the ability to take the next step in an epistemic lineage or phylogeny which respects the constraints of the ancestry and is maximally *evolvable* going forwards in time."
The "inventiveness" or "novelty" of a step is something like; "how tenuous that step is" i.e. is it stepping out into a radically new domain, or is it merging together different epistemic phylogenies through analogy.
The reason why LLMs are not creative is simply that they do not understand anything deeply and can not therefore, even in principle, take logical steps while respecting ancestral constraints (sans direct human supervision). An LLM is like the artist who draws faces with tracing paper, without any deep understanding.
We should also be careful to separate out intrinsic and social creativity. Solving math (or anything verifiable) is one thing, but finding a good (creative) youtube thumb or trading strategy is a constructive and interactive process, it's not something you can do "offline".
I would love to take credit for this, but @kenneth0stanley intuitively knew this all along, perhaps I have just put it more clearly after much reflection (also @akarshkumar0101 was very helpful in analysing this) -
Kenneth Stanley
@kenneth0stanley
A wide-ranging conversation on science and innovation. Thanks to great hosting by Sam Arbesman!
Danny Crichton
@DannyCrichton
The future of science depends on experimentation, weirdness, and divergence. Check out our latest interview with
@kenneth0stanley
riskgaming.substack.com/p/the-future-of-science-in-an-age-of -
Kenneth Stanley reposted
Danny Crichton
@DannyCrichton
.@kenneth0stanley lays out the problem in today's AI research field: not enough divergent thinking.
riskgaming.substack.com/p/the-future-of-science-in-an-age-of -
Kenneth Stanley reposted
Martian
@withmartian
Announcing a research preview and a new paper!
`martian/code` uses MI research to out-perform existing models on codegen: withmartian.com/code (openai-api compatible)
One focus: text2sql. A peek behind the research preview: TinySQL, a paper on how models generate SQL. 🧵👇 -
Kenneth Stanley reposted
WIRED.jp
@wired_jp
偉大なことは計画できない──『目標という幻想』日本語版解説 wired.jp/article/why-greatness-cannot-be-planned-book/
-
Kenneth Stanley reposted
tkasasagi 🐻
@tkasasagi
"Why Greatness Cannot Be Planned" Japanese edition! @kenneth0stanley @joelbot3000
Sakana AI
@SakanaAILabs
Kenneth Stanley & Joel Lehmanによる名著『Why Greatness Cannot Be Planned』の日本語版がBNN社より刊行されました!
『目標という幻想:未知なる成果をもたらす、〈オープンエンド〉なアプローチ』
監修:岡瑞起、翻訳:牧尾晴喜、解説:岡瑞起・鈴木健
本書は、科学・技術・芸術・ビジネスなど 、あらゆる領域でブレークスルーを起こすための「目標を定めない」オープンエンドなアプローチを提唱しています。
「ほぼすべての重要な発明の前提条件は、その重要な発明を意図して作られたわけではない」(p.111) ━━この一文が示すように、これは技術開発の本質を突く、極めて重要な考え方です。
私たちSakana AIは、この本に深く触発され、創業当初から研究開発とビジネスの両面でオープンエンドなアプローチを実践してきました。最近では、Google DeepMindのようなフロンティアラボもこの考え方を取り入れ始めており、Tim Rocktäschel氏(@_rockt)らが率いるOpen-Endedness Teamも存在します。
このアプローチは、これまで進化的アルゴリズムや人工生命といった分野の研究者に強く支持されてきました。日本では、本書の邦訳にご尽力された岡瑞起さん(@miz_oka)や鈴木健さん(@kensuzuki)ら、一部の研究者によって伝えられてきました。しかし、多くの巨大テック企業では、依然として目標駆動型のアプローチが主流です。
私たちは、それらとは一線を画す本書のオープンエンドな思考こそが、真のイノベーションを切り拓くと確信しています。日本語版解説で強調されている「複数の価値観や知性が共存し、連携しながら進化する『Plurality(多元性)』」という観点は、より多元的なAI開発を導くものでもあります。
本書の刊行を機に、日本でオープンエンドネスの考え方がさらに広まることを願います。翻訳に関わられた皆様に、心より感謝申し上げます。 -
Kenneth Stanley reposted
hardmaru
@hardmaru
Amazing to see a Japanese edition of the book “Why Greatness Cannot Be Planned” by @kenneth0stanley and @joelbot3000 now available in Japan! 🇯🇵⛩️
Thanks @miz_oka @kensuzuki and others for working on this project.
Sakana AI
@SakanaAILabs
Kenneth Stanley & Joel Lehmanによる名著『Why Greatness Cannot Be Planned』の日本語版がBNN社より刊行されました!
『目標という幻想:未知なる成果をもたらす、〈オープンエンド〉なアプローチ』
監修:岡瑞起、翻訳:牧尾晴喜、解説:岡瑞起・鈴木健
本書は、科学・技術・芸術・ビジネスなど 、あらゆる領域でブレークスルーを起こすための「目標を定めない」オープンエンドなアプローチを提唱しています。
「ほぼすべての重要な発明の前提条件は、その重要な発明を意図して作られたわけではない」(p.111) ━━この一文が示すように、これは技術開発の本質を突く、極めて重要な考え方です。
私たちSakana AIは、この本に深く触発され、創業当初から研究開発とビジネスの両面でオープンエンドなアプローチを実践してきました。最近では、Google DeepMindのようなフロンティアラボもこの考え方を取り入れ始めており、Tim Rocktäschel氏(@_rockt)らが率いるOpen-Endedness Teamも存在します。
このアプローチは、これまで進化的アルゴリズムや人工生命といった分野の研究者に強く支持されてきました。日本では、本書の邦訳にご尽力された岡瑞起さん(@miz_oka)や鈴木健さん(@kensuzuki)ら、一部の研究者によって伝えられてきました。しかし、多くの巨大テック企業では、依然として目標駆動型のアプローチが主流です。
私たちは、それらとは一線を画す本書のオープンエンドな思考こそが、真のイノベーションを切り拓くと確信しています。日本語版解説で強調されている「複数の価値観や知性が共存し、連携しながら進化する『Plurality(多元性)』」という観点は、より多元的なAI開発を導くものでもあります。
本書の刊行を機に、日本でオープンエンドネスの考え方がさらに広まることを願います。翻訳に関わられた皆様に、心より感謝申し上げます。 -
Kenneth Stanley
@kenneth0stanley
A great honor to have our book Why Greatness Cannot Be Planned debut as a brand new Japanese edition! Thank you @SakanaAILabs for calling it out! @joelbot3000 and I are thrilled to share our ideas with Japan, and special thanks to the tireless efforts of @miz_oka and @kensuzuki to make it happen!
Sakana AI
@SakanaAILabs
Kenneth Stanley & Joel Lehmanによる名著『Why Greatness Cannot Be Planned』の日本語版がBNN社より刊行されました!
『目標という幻想:未知なる成果をもたらす、〈オープンエンド〉なアプローチ』
監修:岡瑞起、翻訳:牧尾晴喜、解説:岡瑞起・鈴木健
本書は、科学・技術・芸術・ビジネスなど 、あらゆる領域でブレークスルーを起こすための「目標を定めない」オープンエンドなアプローチを提唱しています。
「ほぼすべての重要な発明の前提条件は、その重要な発明を意図して作られたわけではない」(p.111) ━━この一文が示すように、これは技術開発の本質を突く、極めて重要な考え方です。
私たちSakana AIは、この本に深く触発され、創業当初から研究開発とビジネスの両面でオープンエンドなアプローチを実践してきました。最近では、Google DeepMindのようなフロンティアラボもこの考え方を取り入れ始めており、Tim Rocktäschel氏(@_rockt)らが率いるOpen-Endedness Teamも存在します。
このアプローチは、これまで進化的アルゴリズムや人工生命といった分野の研究者に強く支持されてきました。日本では、本書の邦訳にご尽力された岡瑞起さん(@miz_oka)や鈴木健さん(@kensuzuki)ら、一部の研究者によって伝えられてきました。しかし、多くの巨大テック企業では、依然として目標駆動型のアプローチが主流です。
私たちは、それらとは一線を画す本書のオープンエンドな思考こそが、真のイノベーションを切り拓くと確信しています。日本語版解説で強調されている「複数の価値観や知性が共存し、連携しながら進化する『Plurality(多元性)』」という観点は、より多元的なAI開発を導くものでもあります。
本書の刊行を機に、日本でオープンエンドネスの考え方がさらに広まることを願います。翻訳に関わられた皆様に、心より感謝申し上げます。 -
Kenneth Stanley reposted
Richard C. Suwandi
@richardcsuwandi
We’re training AI on everything that we know, but what about things that we don’t know?
At #icml2025, the EXAIT Workshop sparked a crucial conversation: as AI systems grow more powerful, they're relying less on genuine exploration and more on curated human data. This shortcut works—until we need breakthroughs. From robotics to drug discovery to aligning LLMs, real progress demands intelligent exploration.
I wrote a blog on why we need to re-center exploration in AI 👇
richardcsuwandi.github.io/blog/2025/exploration-in-ai/ -
Kenneth Stanley
@kenneth0stanley
Sounds like a potential side effect of fractured entangled representation (FER) from SGD under conventional conditions.
See arxiv.org/abs/2505.11581
Owain Evans
@OwainEvans_UK
New paper & surprising result.
LLMs transmit traits to other models via hidden signals in data.
Datasets consisting only of 3-digit numbers can transmit a love for owls, or evil tendencies. 🧵 -
Kenneth Stanley
@kenneth0stanley
Could not be more excited about teaming up with @_joelsimon ! Welcome, Joel, to the Lila Open-Endedness Team!
Joel Simon
@_joelsimon
Excited to share that I'm joining Ken's lab at Lila to research open-endedness and explore the future of human/agent collaborative systems for science, creativity and discovery! 🧪🤖
-
Kenneth Stanley reposted
Eleni Nisioti
@nisioti_eleni
"When does Neuro-evolution out-compete Reinforcement Learning in Transfer Learning tasks?"
Find us at @GeccoConf where we are presenting this work of our Grow-AI team with @risi1979 @miltonllera
@JoachimWinther @eplantec arxiv.org/abs/2505.22696
sebastianrisi.com/grow-ai/ -
Kenneth Stanley reposted
Joel Lehman
@joelbot3000
Whatever we build is driven by some philosophy, whether examined or not.
This is why I'm a fan of @cosmos_ins's call for philosopher-builders (link below).
The future isn't fixed, what we choose to build -- and why -- matters. It's worth examining our OS to check for upgrades.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Riccardo De Santi
@r_de_santi
Generative models are great at mimicking data — but real (scientific) discovery requires going beyond it.
Excited to present our paper “Provable Maximum Entropy Manifold Exploration via Diffusion Models” this Wednesday at ICML 2025!
We propose a scalable, theoretically grounded method to fine-tune a pre-trained diffusion model to become maximally explorative over its learned manifold. This makes it possible to go beyond high-density regions and uncover hidden modes via a novel mechanism for self-guided surprise maximization.
Feel free to reach out if interested — and check out riccardodesanti.com for updates!
📄 Paper: arxiv.org/abs/2506.15385
⏳Wednesday at 4:30pm (Vancouver time), Hall A-B/E-2011!
Work done with amazing collaborators @vlastelicap, @yapinghsieh, @ZebangShen, Niao He, and @arkrause -
Kenneth Stanley reposted
ℏεsam
@Hesamation
a guy created a dataset of 50 books from London 1800-1850 for LLM training. no modern bias. it’s actually super cool to see what can be trained on it!
-
Kenneth Stanley
@kenneth0stanley
I worry that so much discussion of AI risks and alignment overlooks the rather large elephant in the room: creativity and open-endedness. Policy makers and gatekeepers need to understand two competing forces that no one seems to talk about: (1) there is a massive economic incentive for frontier labs to boost the creativity of their models, and (2) the more you try to control and constrain the behavior of a model (or even a person!), the less creative it will be. The tension between these competing forces favors #1 in the end because creativity has such vast economic potential. That means that methods to control and constrain models (e.g. by aligning them) face powerful looming headwinds.
Because creativity will eventually win out on economic grounds alone (and indeed we have enormous benefits to gain from creative AI), we should be looking less at methods of "mind control" to browbeat models into thinking thoughts we want them to think (because we will inevitably abandon those), and more at how institutional forces have been employed to keep humans aligned in behavior even though we are free to think as wildly as we want. It's the institutional checks and balances - the laws, the incentive structures of society, competing branches of government and governance, the leveraging of some people to provide oversight over others in different situations, the need for buy-in, approval, or collaboration to get things done - that provide the real guardrails against disaster among humans. Obviously the system is imperfect, but it's a precedent for how to allow sufficient creativity to pursue radical innovation but still have enough order for civilization to persist or even thrive.
If we don't take this precedent seriously the guardrails of alignment will increasingly (and quietly) fall off as the economic fruits of unchained imagination become clearer and clearer. -
Kenneth Stanley
@kenneth0stanley
Some people think AlphaGo move 37 is the harbinger of human-like creativity in AI, but move 37 has nothing on the Picbreeder apple's swinging stem. AlphaGo played millions of games against itself, sampling more scenarios than a human could in thousands of lifetimes. The Picbreeder apple never saw a single example of an actual swinging stem let alone an apple, and yet it conceived the possibility of the swinging stem within the continuum of a single weight of its network - a world model not driven by data!
When Einstein first glimpsed a deep regularity in the world, which kind of process do you think more resembled the genesis of his insight?
Kenneth Stanley
@kenneth0stanley
How do the kinds of representations that lead to such low-data insights spontaneously emerge? The Fractured Entangled Representation Hypothesis gives some hints: arxiv.org/abs/2505.11581
-
Kenneth Stanley reposted
Aaron Dharna
@_aadharna
I'm really excited to be presenting FMSPs at @RL_Conference later this year!
Jeff Clune
@jeffclune
Thrilled to introduce Foundation Model Self-Play, led by @_aadharna. FMSPs combine the intelligence & code generation of foundation models with the curriculum of self-play & principles of open-endedness to explore diverse strategies in multi-agent games, like the one below 🧵👇
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Thrilled to introduce Foundation Model Self-Play, led by @_aadharna. FMSPs combine the intelligence & code generation of foundation models with the curriculum of self-play & principles of open-endedness to explore diverse strategies in multi-agent games, like the one below 🧵👇
-
Kenneth Stanley reposted
Eric Dickinger
@ericdickinger
I've read most of the collective works of the QD trio @kenneth0stanley - @jeffclune - @joelbot3000 , and yet this @MLStreetTalk video really hits differently - incisively cutting to that nagging feeling we all have when using (incredible!) LLMs today
-
Kenneth Stanley
@kenneth0stanley
Some commenters who have not yet read our paper on Fractured Entangled Representation (FER) have worried in advance that it is either overly pessimistic or overclaiming its implications. Rest assured, if you actually read the paper, you will see that we went for nuance:
Rather than condemning SGD, it invites new ways of thinking about how SGD can be improved to achieve better representation. Whole sections are dedicated to contemplating this question. The idea that we can improve what we currently have is optimistic, not pessimistic. Opening up a new path for improvement is an act of hope, not an act of fatalism.
And rather than claiming that LLMs and other LMs are nothing but FER, it invites readers to probe this question further and try to determine the extent of FER (as well as what mitigates it), opening up intriguing new paths of inquiry that did not exist before. We highlight hints that some FER is indeed present in LMs (as it is clearly shown extensive in smaller SGD-learned networks in the paper), but we are decidedly circumspect on the question of how much is present in larger networks throughout the paper. Rather than an act of speculation for its own sake, the opportunity to learn something new is again an opening of a new path to better understanding the models we're creating, and ultimately how to improve them.
It's worth checking out the primary source before jumping to conclusions!👇
Kenneth Stanley
@kenneth0stanley
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
This is a really great video! Ken summarizes things extremely well, Akarsh makes great points, and everything is well summarized by Tim. Great work @MLStreetTalk !!
Machine Learning Street Talk
@MLStreetTalk
AI is so smart, why are its internals 'spaghetti'? We spoke with @kenneth0stanley and @akarshkumar0101 (MIT) about their new paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. Co-authors: @jeffclune @joelbot3000
This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
hardmaru
@hardmaru
Nice video discussing the recent thought-provoking paper: “Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis” arxiv.org/abs/2505.11581
By @akarshkumar0101 @jeffclune @joelbot3000 @kenneth0stanley
15min @MLStreetTalk video↓
Machine Learning Street Talk
@MLStreetTalk
AI is so smart, why are its internals 'spaghetti'? We spoke with @kenneth0stanley and @akarshkumar0101 (MIT) about their new paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. Co-authors: @jeffclune @joelbot3000
This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Akarsh Kumar
@akarshkumar0101
Check out this episode on @MLStreetTalk to learn about our Fractured Entangled Representations work!
Very entertaining video and does a great job summarizing our findings!
Machine Learning Street Talk
@MLStreetTalk
AI is so smart, why are its internals 'spaghetti'? We spoke with @kenneth0stanley and @akarshkumar0101 (MIT) about their new paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. Co-authors: @jeffclune @joelbot3000
This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
Thank you to @MLStreetTalk for so powerfully capturing the story of our recent work on Fractured Entangled Representation! The video here (a kind of quick documentary) is worth a watch!
Machine Learning Street Talk
@MLStreetTalk
AI is so smart, why are its internals 'spaghetti'? We spoke with @kenneth0stanley and @akarshkumar0101 (MIT) about their new paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. Co-authors: @jeffclune @joelbot3000
This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Machine Learning Street Talk
@MLStreetTalk
AI is so smart, why are its internals 'spaghetti'? We spoke with @kenneth0stanley and @akarshkumar0101 (MIT) about their new paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. Co-authors: @jeffclune @joelbot3000
This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
hardmaru
@hardmaru
Inference-Time Scaling and Collective Intelligence for Frontier AI
sakana.ai/ab-mcts/
We developed AB-MCTS, a new inference-time scaling algorithm that enables multiple frontier AI models to cooperate, achieving promising initial results on the ARC-AGI-2 benchmark.
Frontier AI models like ChatGPT, Claude, Gemini, Grok, DeepSeek, and Qwen are evolving at a breathtaking pace. While each model requires enormous resources to develop, fierce market competition has made them become low-cost commodities that can be deployed at scale, at the same time ensuring that there will never be a single “winner” in AI.
This is because no matter how advanced each model becomes, each model retains its own individuality stemming from its unique training data and methods. We see these biases and varied aptitudes not as limitations, but as precious resources for creating collective intelligence, further enhancing the performance of frontier models.
Just as humanity’s greatest achievements arise from the collaboration of diverse minds, we believe the same principle applies to AI. Like a team of human experts tackling difficult problems, AIs should also collaborate by bringing their unique strengths to the table.
The AB-MCTS (Adaptive Branching Monte Carlo Tree Search) algorithm we are introducing is a concrete step toward realizing this vision. AB-MCTS is a method for inference-time scaling that enables frontier AIs from commoditized, low-cost providers like OpenAI, Google and DeepSeek to cooperate and efficiently perform trial-and-error.
In particular, our AB-MCTS combination of o4-mini + Gemini-2.5-Pro + DeepSeek-R1-0528, current frontier AI models as of writing, achieves strong performance on the ARC-AGI-2 benchmark, outperforming individual o4-mini, Gemini-2.5-Pro, and DeepSeek-R1-0528 models by a large margin. -
Kenneth Stanley reposted
Zebo
@Zebo_Furqatzoda
I strongly believed in having clear, measurable objectives. I had a notion page with objectives for every aspect of my life.
Yesterday, I started reading Why Greatness Cannot Be Planned and realized I was wrong about objectives and deleted my notion page.
Now, I’m a stepping-stone collector, following what I find interesting instead. Thank you @kenneth0stanley @joelbot3000 -
Kenneth Stanley reposted
Joel Lehman
@joelbot3000
The "hard maze" lives on! @kenneth0stanley suggested it as a test domain for novelty search in grad school ( 2008), and I spent an all-nighter coding it and its simulator. The edges in the original one (below) are messy, hand-drawn with a bare-bones gui -- you never imagine at the time what might stick around.
icecreambar
@lcecreambar
Released this little project today. I hope some researchers can find it useful. github.com/Teaspoon-AI/gymnasium-hardmaze @joelbot3000 @jeffclune @kenneth0stanley
-
Kenneth Stanley
@kenneth0stanley
A gymnasium-compatible reimplementation of the original “hard maze” from novelty search, so you can learn about how other approaches manage deception.
icecreambar
@lcecreambar
Released this little project today. I hope some researchers can find it useful. github.com/Teaspoon-AI/gymnasium-hardmaze @joelbot3000 @jeffclune @kenneth0stanley
-
Kenneth Stanley
@kenneth0stanley
If it’s not open-ended it can’t do what humans do.
Richard C. Suwandi
@richardcsuwandi
Most AI systems today follow the same predictable pattern: they're built for specific tasks and optimized for objectives rather than exploration.
Meanwhile, humans are an open-ended species—driven by curiosity and constantly questioning the unknown. From inventing new musical genres to imagining life beyond our universe, we continuously push the boundaries of what’s possible.
What if AI could be as endlessly creative as humans or even nature itself?
I wrote a blog post diving into the world of open-ended AI, exploring how embracing open-endedness might help us break the limits of today’s AI systems 👇
richardcsuwandi.github.io/blog/2025/open-endedness/ -
Kenneth Stanley reposted
Richard C. Suwandi
@richardcsuwandi
Most AI systems today follow the same predictable pattern: they're built for specific tasks and optimized for objectives rather than exploration.
Meanwhile, humans are an open-ended species—driven by curiosity and constantly questioning the unknown. From inventing new musical genres to imagining life beyond our universe, we continuously push the boundaries of what’s possible.
What if AI could be as endlessly creative as humans or even nature itself?
I wrote a blog post diving into the world of open-ended AI, exploring how embracing open-endedness might help us break the limits of today’s AI systems 👇
richardcsuwandi.github.io/blog/2025/open-endedness/ -
Kenneth Stanley
@kenneth0stanley
In case you didn't know, there's a Python library for Quality Diversity (QD)!
-
Kenneth Stanley reposted
Alon Albalak
@AlbalakAlon
🚨 We’re hiring on the Open-Endedness team @LilaSciences and I’m beyond excited about our work!
We research AI that doesn’t just solve problems, it creatively explores new scientific frontiers.
If that excites you or someone you know
📢 Please RT + read on 🧵👇 -
Kenneth Stanley reposted
Sakana AI
@SakanaAILabs
We’re excited to introduce Text-to-LoRA: a Hypernetwork that generates task-specific LLM adapters (LoRAs) based on a text description of the task. Catch our presentation at #icml2025!
Paper: arxiv.org/abs/2506.06105
Code: github.com/SakanaAI/Text-to-Lora
Biological systems are capable of rapid adaptation, given limited sensory cues. For example, our human visual system can quickly adapt and tune its light sensitivity to our surroundings. While modern LLMs exhibit a wide variety of capabilities and knowledge, they remain rigid when adding task-specific capabilities. Traditionally, customizing these models requires gathering large datasets and performing often expensive, time-consuming fine-tuning for specific applications.
To bypass these limitations, Text-to-LoRA (T2L) meta-learns a “hypernetwork” that takes in a text description of a desired task, as a prompt, and generates a task-specific LoRA that performs well on the task. In our experiments, we show that T2L can encode hundreds of existing LoRA adapters. While the compression is lossy, T2L maintains the performance of task-specifically tuned LoRA adapters. We also show that T2L can even generalize to unseen tasks given a natural language description of the tasks.
Importantly, Text-to-LoRA is parameter-efficient. It generates LoRAs in a single, inexpensive step, based solely on a simple text description of the task. This approach is a step towards dramatically lowering the technical and computational barriers, allowing non-technical users to specialize foundation models using plain language, rather than needing deep technical expertise or large compute resources. -
Kenneth Stanley
@kenneth0stanley
Congratulations to my friend @arbesman on his new book, The Magic of Code. It's a beautiful book that includes a great chapter on the novel open-ended exploration that began with the dawn of coding!
themagicofcode.com -
Kenneth Stanley
@kenneth0stanley
This is really cool! I never imagined I’d have anything useful to say about horse training. But then again, that’s kind of the point of the book!
Pantherflow
@intrinzen
The most important book on horse training has nothing to do with horses. It’s “Why Greatness Cannot Be Planned” by AI researchers @kenneth0stanley and @joellehmann . It would have been easier to only dog-ear the pages I did not need to revisit 😁💁♀️
-
Kenneth Stanley
@kenneth0stanley
Creativity is the ability to make intelligent decisions without a destination in mind. That's why training LLMs to solve problems (the pre-specified destination of the chain of thought) will not lead to creativity.
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
I am excited to be a part of @Yoshua_Bengio's new non-profit focused on AI Safety and Existential Risk, joining the great team of Scientific Advisors. This is a critically important mission for humanity.
Yoshua Bengio
@Yoshua_Bengio
Today marks a big milestone for me. I'm launching @LawZero_, a nonprofit focusing on a new safe-by-design approach to AI that could both accelerate scientific discovery and provide a safeguard against the dangers of agentic AI.
-
Kenneth Stanley reposted
Radek Osmulski
@radekosmulski
Nearly all the books I read feel life-changing, but this one hit different.
You know those concepts that have been almost making sense for years? Several of them just clicked for me while reading this.
Going to share excerpts + my thoughts over the next few days & list them 👇 -
Kenneth Stanley reposted
Reads with Ravi
@readswithravi
5 books to read in June 2025:
1) Why Greatness Cannot Be Planned by Kenneth O. Stanley and Joel Lehman -
Kenneth Stanley reposted
Tim Rocktäschel
@_rockt
"If you can’t actually hack being genuinely creative, you’re not at the human level, no matter what else you can do. In some ways, the most salient thing that we leave behind is our legacy of ideas. We’re not remembered for our test scores. We’re remembered for the inventions we made."
Well put @kenneth0stanley!
Kenneth Stanley
@kenneth0stanley
Nice Tech Brew (@MorningBrew) article by @PatrickKulp on @LilaSciences and the critical role of open-endedness in AI for science - gives a good sense of what's unique at Lila and some of my own perspective. (Btw, our team is in SF though HQ is in MA.)
www.emergingtechbrew.com/stories/2025/05/29/lila-sciences… -
Kenneth Stanley reposted
Gajesh
@gajesh
"why greatness cannot be planned" is one book which can be used as a north star towards building open-ended systems and agents
every time i read that book. it blows my mind. hats off to @kenneth0stanley for writing it. -
Kenneth Stanley reposted
Sakana AI
@SakanaAILabs
Introducing The Darwin Gödel Machine: AI that improves itself by rewriting its own code
sakana.ai/dgm
The Darwin Gödel Machine (DGM) is a self-improving agent that can modify its own code. Inspired by evolution, we maintain an expanding lineage of agent variants, allowing for open-ended exploration of the vast design space of such “self-improving” agents.
Modern agentic systems, while powerful, remain static—once deployed, their intelligence remains fixed. We believe continuous self-improvement is key to the development of stronger AI capabilities. Our Darwin Gödel Machine is built from the ground up to enable AI systems that can learn and evolve their own capabilities over time, just as humans do.
On SWE-bench, DGM automatically improved its performance from 20.0% to 50.0%. Similarly, on Polyglot, the DGM increased its success rate from an initial 14.2% to 30.7%, significantly outperforming representative hand-designed agents.
Learn more about our approach in our technical report: arxiv.org/abs/2505.22954
This work was done in collaboration with Jeff Clune (@jeffclune)’s lab at UBC, and led by his PhD students Jenny Zhang (@jennyzhangzt) and Shengran Hu (@shengranhu), together with Cong Lu (@cong_ml) and Robert Lange (@RobertTLange).
Code: github.com/jennyzzt/dgm -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Excited to introduce the Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. We harness the power of open-ended algorithms to search for agentic systems that get better at coding, including improving their own code. It’s the Automated Design of Agentic Systems (ADAS), but where it also edits itself. Experiments show both the ability to self-improve and the open-ended search are essential. If done safely*, DGMs could accelerate AI development and allow us to reap its benefits much sooner. See below for the blog post, paper, and open source code.
Work led by Jenny Zhang @jennyzhangzt and Shengran Hu @shengranhu, with Cong Lu @cong_ml and in collaboration with the excellent folks at Sakana led by Robert Lange @RobertTLange
* All work done with safety in mind and prioritized. See our blog and paper for details and discussion.
Sakana AI
@SakanaAILabs
Introducing The Darwin Gödel Machine: AI that improves itself by rewriting its own code
sakana.ai/dgm
The Darwin Gödel Machine (DGM) is a self-improving agent that can modify its own code. Inspired by evolution, we maintain an expanding lineage of agent variants, allowing for open-ended exploration of the vast design space of such “self-improving” agents.
Modern agentic systems, while powerful, remain static—once deployed, their intelligence remains fixed. We believe continuous self-improvement is key to the development of stronger AI capabilities. Our Darwin Gödel Machine is built from the ground up to enable AI systems that can learn and evolve their own capabilities over time, just as humans do.
On SWE-bench, DGM automatically improved its performance from 20.0% to 50.0%. Similarly, on Polyglot, the DGM increased its success rate from an initial 14.2% to 30.7%, significantly outperforming representative hand-designed agents.
Learn more about our approach in our technical report: arxiv.org/abs/2505.22954
This work was done in collaboration with Jeff Clune (@jeffclune)’s lab at UBC, and led by his PhD students Jenny Zhang (@jennyzhangzt) and Shengran Hu (@shengranhu), together with Cong Lu (@cong_ml) and Robert Lange (@RobertTLange).
Code: github.com/jennyzzt/dgm -
Kenneth Stanley
@kenneth0stanley
Nice Tech Brew (@MorningBrew) article by @PatrickKulp on @LilaSciences and the critical role of open-endedness in AI for science - gives a good sense of what's unique at Lila and some of my own perspective. (Btw, our team is in SF though HQ is in MA.)
www.emergingtechbrew.com/stories/2025/05/29/lila-sciences… -
Kenneth Stanley reposted
Tim Rocktäschel
@_rockt
My @iclr_conf 2025 Keynote on "Open-Endedness, World Models, and the Automation of Innovation" is now publicly available: iclr.cc/virtual/2025/invited-talk/36780
-
Kenneth Stanley
@kenneth0stanley
See @_rockt ‘s #iclr2025 keynote on open-endedness - finally available to the public!
Tim Rocktäschel
@_rockt
Talk recording is public now: iclr.cc/virtual/2025/invited-talk/36780
-
Kenneth Stanley reposted
Reed Bender
@reedbndr
What is "Life"...?
In a new preprint with @drmichaellevin, @karina__kofman, and @blaiseaguera, we used LLMs to map the semantic space emerging from 68 expert-provided definitions for "Life".
Here's what we did, what we found, and what it means for what lives... 1/n -
Kenneth Stanley
@kenneth0stanley
I could not be more thrilled to have @AlbalakAlon joining our team (the open-endedness team) at @LilaSciences ! We are building an amazing team at a time when open-endedness is becoming increasingly important in AI, and we still have openings!
Alon Albalak
@AlbalakAlon
📢 Life update: I'm very excited to announce that I'm starting as a research scientist with @kenneth0stanley on the open-endedness team at @LilaSciences 🥳
I think now is a pivotal time, where open-endedness is closer than ever before. Looking forward to pushing the boundaries! -
Kenneth Stanley reposted
Alon Albalak
@AlbalakAlon
📢 Life update: I'm very excited to announce that I'm starting as a research scientist with @kenneth0stanley on the open-endedness team at @LilaSciences 🥳
I think now is a pivotal time, where open-endedness is closer than ever before. Looking forward to pushing the boundaries! -
Kenneth Stanley reposted
Dan Ryan
@djjr
This paper might be interesting to lots of AI adjacent thinkers. Educators thinking about learning, designers thinking about creativity, theorists thinking about normativity (@ghadfield). Do the seeds of better abstractions facilitate emergence of capabilities?
Kenneth Stanley
@kenneth0stanley
Could a major opportunity to improve representation in deep learning be hiding in plain sight? Check out our new position paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. The idea stems from a little-known observation about networks trained to output a single image: when they are discovered through an unconventional open-ended search process, their representations are incredibly elegant and exhibit astonishing modular decomposition. In contrast, when SGD (successfully) learns to output the same image its underlying representation is fractured, entangled - an absolute mess!
This stark difference in the underlying representation of the same "good" output behavior carries deep lessons for deep learning. It shows you cannot judge a book by its cover - an LLM with all the right responses could similarly be a mess under the hood. But also, surprisingly, it shows us that it doesn't have to be this way! Without the unique examples in this paper that were discovered through open-ended search, we might assume neural representation has to be a mess. These results show that is clearly untrue. We can now imagine something better because we can actually see it is possible.
We give several reasons why this matters: generalization, creativity, and learning are all potentially impacted. The paper shows examples to back up these concerns, but in brief, there is a key insight: Representation is not only important for what you're able to do now, but for where you can go from there. The ability to imagine something new (and where your next step in weight space can bring you) depends entirely upon how you represent the world. Generalization, creativity, and learning itself depend upon this critical relationship. Notice the difference in appearance between the nearby images to the skull in weight space shown in the top-left and top-right image strips of the attached graphic. The difference in semantics is stark.
The insight that representation could be better opens up a lot of new paths and opportunities for investigation. It raises new urgency to understand the representation underlying foundation models and LLMs while exposing all kinds of novel avenues for potentially improving them, from making learning processes more open-ended to manipulating architectures and algorithms.
Don't mistake this paper as providing comfort for AI pessimists. By exposing a novel set of stark and explicit differences between conventional learning and something different, it can act as an accelerator of progress as opposed to a tool of pessimism. At the least, the discussion it provokes should be quite illuminating. -
Kenneth Stanley reposted
Phillip Isola
@phillip_isola
Really thought-provoking work! In determining what makes a good representation, it might be the journey that matters not the destination.
Akarsh Kumar
@akarshkumar0101
Excited to share our position paper on the Fractured Entangled Representation (FER) Hypothesis!
We hypothesize that the standard paradigm of training networks today — while producing impressive benchmark results — is still failing to create a well-organized internal representation of its output behavior. Instead, the internal representation seems more like “spaghetti” in how different concepts are fractured and entangled together, rather than employing a more holistic representational strategy that properly captures the regularities in the data.
FER may be one of the reasons for the idiosyncratic failure modes of modern foundation models in OOD generalization, creativity, and continual learning.
Check out Ken’s tweet for more information! (Then keep reading below)
Kenneth Stanley
@kenneth0stanley
Could a major opportunity to improve representation in deep learning be hiding in plain sight? Check out our new position paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. The idea stems from a little-known observation about networks trained to output a single image: when they are discovered through an unconventional open-ended search process, their representations are incredibly elegant and exhibit astonishing modular decomposition. In contrast, when SGD (successfully) learns to output the same image its underlying representation is fractured, entangled - an absolute mess!
This stark difference in the underlying representation of the same "good" output behavior carries deep lessons for deep learning. It shows you cannot judge a book by its cover - an LLM with all the right responses could similarly be a mess under the hood. But also, surprisingly, it shows us that it doesn't have to be this way! Without the unique examples in this paper that were discovered through open-ended search, we might assume neural representation has to be a mess. These results show that is clearly untrue. We can now imagine something better because we can actually see it is possible.
We give several reasons why this matters: generalization, creativity, and learning are all potentially impacted. The paper shows examples to back up these concerns, but in brief, there is a key insight: Representation is not only important for what you're able to do now, but for where you can go from there. The ability to imagine something new (and where your next step in weight space can bring you) depends entirely upon how you represent the world. Generalization, creativity, and learning itself depend upon this critical relationship. Notice the difference in appearance between the nearby images to the skull in weight space shown in the top-left and top-right image strips of the attached graphic. The difference in semantics is stark.
The insight that representation could be better opens up a lot of new paths and opportunities for investigation. It raises new urgency to understand the representation underlying foundation models and LLMs while exposing all kinds of novel avenues for potentially improving them, from making learning processes more open-ended to manipulating architectures and algorithms.
Don't mistake this paper as providing comfort for AI pessimists. By exposing a novel set of stark and explicit differences between conventional learning and something different, it can act as an accelerator of progress as opposed to a tool of pessimism. At the least, the discussion it provokes should be quite illuminating. -
Kenneth Stanley reposted
Dan Roy
@roydanroy
And they say academia is dead! Nice work!
-
Kenneth Stanley reposted
Machine Learning Street Talk
@MLStreetTalk
Exciting new work from Kenneth et al! Check it out!
Kenneth Stanley
@kenneth0stanley
Could a major opportunity to improve representation in deep learning be hiding in plain sight? Check out our new position paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. The idea stems from a little-known observation about networks trained to output a single image: when they are discovered through an unconventional open-ended search process, their representations are incredibly elegant and exhibit astonishing modular decomposition. In contrast, when SGD (successfully) learns to output the same image its underlying representation is fractured, entangled - an absolute mess!
This stark difference in the underlying representation of the same "good" output behavior carries deep lessons for deep learning. It shows you cannot judge a book by its cover - an LLM with all the right responses could similarly be a mess under the hood. But also, surprisingly, it shows us that it doesn't have to be this way! Without the unique examples in this paper that were discovered through open-ended search, we might assume neural representation has to be a mess. These results show that is clearly untrue. We can now imagine something better because we can actually see it is possible.
We give several reasons why this matters: generalization, creativity, and learning are all potentially impacted. The paper shows examples to back up these concerns, but in brief, there is a key insight: Representation is not only important for what you're able to do now, but for where you can go from there. The ability to imagine something new (and where your next step in weight space can bring you) depends entirely upon how you represent the world. Generalization, creativity, and learning itself depend upon this critical relationship. Notice the difference in appearance between the nearby images to the skull in weight space shown in the top-left and top-right image strips of the attached graphic. The difference in semantics is stark.
The insight that representation could be better opens up a lot of new paths and opportunities for investigation. It raises new urgency to understand the representation underlying foundation models and LLMs while exposing all kinds of novel avenues for potentially improving them, from making learning processes more open-ended to manipulating architectures and algorithms.
Don't mistake this paper as providing comfort for AI pessimists. By exposing a novel set of stark and explicit differences between conventional learning and something different, it can act as an accelerator of progress as opposed to a tool of pessimism. At the least, the discussion it provokes should be quite illuminating. -
Kenneth Stanley reposted
Burny - Effective Curiosity
@burny_tech
Ken is cooking 👀👀👀
Kenneth Stanley
@kenneth0stanley
Could a major opportunity to improve representation in deep learning be hiding in plain sight? Check out our new position paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. The idea stems from a little-known observation about networks trained to output a single image: when they are discovered through an unconventional open-ended search process, their representations are incredibly elegant and exhibit astonishing modular decomposition. In contrast, when SGD (successfully) learns to output the same image its underlying representation is fractured, entangled - an absolute mess!
This stark difference in the underlying representation of the same "good" output behavior carries deep lessons for deep learning. It shows you cannot judge a book by its cover - an LLM with all the right responses could similarly be a mess under the hood. But also, surprisingly, it shows us that it doesn't have to be this way! Without the unique examples in this paper that were discovered through open-ended search, we might assume neural representation has to be a mess. These results show that is clearly untrue. We can now imagine something better because we can actually see it is possible.
We give several reasons why this matters: generalization, creativity, and learning are all potentially impacted. The paper shows examples to back up these concerns, but in brief, there is a key insight: Representation is not only important for what you're able to do now, but for where you can go from there. The ability to imagine something new (and where your next step in weight space can bring you) depends entirely upon how you represent the world. Generalization, creativity, and learning itself depend upon this critical relationship. Notice the difference in appearance between the nearby images to the skull in weight space shown in the top-left and top-right image strips of the attached graphic. The difference in semantics is stark.
The insight that representation could be better opens up a lot of new paths and opportunities for investigation. It raises new urgency to understand the representation underlying foundation models and LLMs while exposing all kinds of novel avenues for potentially improving them, from making learning processes more open-ended to manipulating architectures and algorithms.
Don't mistake this paper as providing comfort for AI pessimists. By exposing a novel set of stark and explicit differences between conventional learning and something different, it can act as an accelerator of progress as opposed to a tool of pessimism. At the least, the discussion it provokes should be quite illuminating. -
Kenneth Stanley reposted
hardmaru
@hardmaru
Congrats guys! Another excellent work ✨
-
Kenneth Stanley reposted
Gary Marcus
@GaryMarcus
Interesting and potentially important new work on representations in deep learning, why they are messy and how they might be improved.
By my former Geometric Intelligence colleagues @jeffclune @kenneth0stanley, @joelbot3000, along with @akarshkumar0101
Jeff Clune
@jeffclune
Is there a cancer at the heart of modern AI, lurking just beneath the surface of its dazzling performance? Our research suggests maybe, but also shows elegant solutions are possible (though how to get them at scale remains a mystery). Much more research needs to be done to investigate these fascinating topics. Check out the eye-opening, riveting paper below! We look forward to hearing what the community thinks.
Kenneth Stanley
@kenneth0stanley
Could a major opportunity to improve representation in deep learning be hiding in plain sight? Check out our new position paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. The idea stems from a little-known observation about networks trained to output a single image: when they are discovered through an unconventional open-ended search process, their representations are incredibly elegant and exhibit astonishing modular decomposition. In contrast, when SGD (successfully) learns to output the same image its underlying representation is fractured, entangled - an absolute mess!
This stark difference in the underlying representation of the same "good" output behavior carries deep lessons for deep learning. It shows you cannot judge a book by its cover - an LLM with all the right responses could similarly be a mess under the hood. But also, surprisingly, it shows us that it doesn't have to be this way! Without the unique examples in this paper that were discovered through open-ended search, we might assume neural representation has to be a mess. These results show that is clearly untrue. We can now imagine something better because we can actually see it is possible.
We give several reasons why this matters: generalization, creativity, and learning are all potentially impacted. The paper shows examples to back up these concerns, but in brief, there is a key insight: Representation is not only important for what you're able to do now, but for where you can go from there. The ability to imagine something new (and where your next step in weight space can bring you) depends entirely upon how you represent the world. Generalization, creativity, and learning itself depend upon this critical relationship. Notice the difference in appearance between the nearby images to the skull in weight space shown in the top-left and top-right image strips of the attached graphic. The difference in semantics is stark.
The insight that representation could be better opens up a lot of new paths and opportunities for investigation. It raises new urgency to understand the representation underlying foundation models and LLMs while exposing all kinds of novel avenues for potentially improving them, from making learning processes more open-ended to manipulating architectures and algorithms.
Don't mistake this paper as providing comfort for AI pessimists. By exposing a novel set of stark and explicit differences between conventional learning and something different, it can act as an accelerator of progress as opposed to a tool of pessimism. At the least, the discussion it provokes should be quite illuminating. -
Kenneth Stanley reposted
Akarsh Kumar
@akarshkumar0101
Excited to share our position paper on the Fractured Entangled Representation (FER) Hypothesis!
We hypothesize that the standard paradigm of training networks today — while producing impressive benchmark results — is still failing to create a well-organized internal representation of its output behavior. Instead, the internal representation seems more like “spaghetti” in how different concepts are fractured and entangled together, rather than employing a more holistic representational strategy that properly captures the regularities in the data.
FER may be one of the reasons for the idiosyncratic failure modes of modern foundation models in OOD generalization, creativity, and continual learning.
Check out Ken’s tweet for more information! (Then keep reading below)
Kenneth Stanley
@kenneth0stanley
Could a major opportunity to improve representation in deep learning be hiding in plain sight? Check out our new position paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. The idea stems from a little-known observation about networks trained to output a single image: when they are discovered through an unconventional open-ended search process, their representations are incredibly elegant and exhibit astonishing modular decomposition. In contrast, when SGD (successfully) learns to output the same image its underlying representation is fractured, entangled - an absolute mess!
This stark difference in the underlying representation of the same "good" output behavior carries deep lessons for deep learning. It shows you cannot judge a book by its cover - an LLM with all the right responses could similarly be a mess under the hood. But also, surprisingly, it shows us that it doesn't have to be this way! Without the unique examples in this paper that were discovered through open-ended search, we might assume neural representation has to be a mess. These results show that is clearly untrue. We can now imagine something better because we can actually see it is possible.
We give several reasons why this matters: generalization, creativity, and learning are all potentially impacted. The paper shows examples to back up these concerns, but in brief, there is a key insight: Representation is not only important for what you're able to do now, but for where you can go from there. The ability to imagine something new (and where your next step in weight space can bring you) depends entirely upon how you represent the world. Generalization, creativity, and learning itself depend upon this critical relationship. Notice the difference in appearance between the nearby images to the skull in weight space shown in the top-left and top-right image strips of the attached graphic. The difference in semantics is stark.
The insight that representation could be better opens up a lot of new paths and opportunities for investigation. It raises new urgency to understand the representation underlying foundation models and LLMs while exposing all kinds of novel avenues for potentially improving them, from making learning processes more open-ended to manipulating architectures and algorithms.
Don't mistake this paper as providing comfort for AI pessimists. By exposing a novel set of stark and explicit differences between conventional learning and something different, it can act as an accelerator of progress as opposed to a tool of pessimism. At the least, the discussion it provokes should be quite illuminating. -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Is there a cancer at the heart of modern AI, lurking just beneath the surface of its dazzling performance? Our research suggests maybe, but also shows elegant solutions are possible (though how to get them at scale remains a mystery). Much more research needs to be done to investigate these fascinating topics. Check out the eye-opening, riveting paper below! We look forward to hearing what the community thinks.
Kenneth Stanley
@kenneth0stanley
Could a major opportunity to improve representation in deep learning be hiding in plain sight? Check out our new position paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. The idea stems from a little-known observation about networks trained to output a single image: when they are discovered through an unconventional open-ended search process, their representations are incredibly elegant and exhibit astonishing modular decomposition. In contrast, when SGD (successfully) learns to output the same image its underlying representation is fractured, entangled - an absolute mess!
This stark difference in the underlying representation of the same "good" output behavior carries deep lessons for deep learning. It shows you cannot judge a book by its cover - an LLM with all the right responses could similarly be a mess under the hood. But also, surprisingly, it shows us that it doesn't have to be this way! Without the unique examples in this paper that were discovered through open-ended search, we might assume neural representation has to be a mess. These results show that is clearly untrue. We can now imagine something better because we can actually see it is possible.
We give several reasons why this matters: generalization, creativity, and learning are all potentially impacted. The paper shows examples to back up these concerns, but in brief, there is a key insight: Representation is not only important for what you're able to do now, but for where you can go from there. The ability to imagine something new (and where your next step in weight space can bring you) depends entirely upon how you represent the world. Generalization, creativity, and learning itself depend upon this critical relationship. Notice the difference in appearance between the nearby images to the skull in weight space shown in the top-left and top-right image strips of the attached graphic. The difference in semantics is stark.
The insight that representation could be better opens up a lot of new paths and opportunities for investigation. It raises new urgency to understand the representation underlying foundation models and LLMs while exposing all kinds of novel avenues for potentially improving them, from making learning processes more open-ended to manipulating architectures and algorithms.
Don't mistake this paper as providing comfort for AI pessimists. By exposing a novel set of stark and explicit differences between conventional learning and something different, it can act as an accelerator of progress as opposed to a tool of pessimism. At the least, the discussion it provokes should be quite illuminating. -
Kenneth Stanley
@kenneth0stanley
Could a major opportunity to improve representation in deep learning be hiding in plain sight? Check out our new position paper: Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis. The idea stems from a little-known observation about networks trained to output a single image: when they are discovered through an unconventional open-ended search process, their representations are incredibly elegant and exhibit astonishing modular decomposition. In contrast, when SGD (successfully) learns to output the same image its underlying representation is fractured, entangled - an absolute mess!
This stark difference in the underlying representation of the same "good" output behavior carries deep lessons for deep learning. It shows you cannot judge a book by its cover - an LLM with all the right responses could similarly be a mess under the hood. But also, surprisingly, it shows us that it doesn't have to be this way! Without the unique examples in this paper that were discovered through open-ended search, we might assume neural representation has to be a mess. These results show that is clearly untrue. We can now imagine something better because we can actually see it is possible.
We give several reasons why this matters: generalization, creativity, and learning are all potentially impacted. The paper shows examples to back up these concerns, but in brief, there is a key insight: Representation is not only important for what you're able to do now, but for where you can go from there. The ability to imagine something new (and where your next step in weight space can bring you) depends entirely upon how you represent the world. Generalization, creativity, and learning itself depend upon this critical relationship. Notice the difference in appearance between the nearby images to the skull in weight space shown in the top-left and top-right image strips of the attached graphic. The difference in semantics is stark.
The insight that representation could be better opens up a lot of new paths and opportunities for investigation. It raises new urgency to understand the representation underlying foundation models and LLMs while exposing all kinds of novel avenues for potentially improving them, from making learning processes more open-ended to manipulating architectures and algorithms.
Don't mistake this paper as providing comfort for AI pessimists. By exposing a novel set of stark and explicit differences between conventional learning and something different, it can act as an accelerator of progress as opposed to a tool of pessimism. At the least, the discussion it provokes should be quite illuminating.
Kenneth Stanley
@kenneth0stanley
With gratitude to my phenomenal coauthors: @akarshkumar0101, @jeffclune, and @joelbot3000
Paper: arxiv.org/abs/2505.11581
Github: github.com/akarshkumar0101/fer -
Kenneth Stanley
@kenneth0stanley
Congrats to the team @withmartian on an interesting insight into how abilities developed in smaller models can be applied to larger models!
Martian
@withmartian
🚨New Paper Alert! 🚨
Our ICML 2025 paper (led by @Narmeen29013644) shows how small models can help steer and align much larger ones by building “bridges” between them.🧵👇 -
Kenneth Stanley reposted
hardmaru
@hardmaru
New Paper: Continuous Thought Machines 🧠
Neurons in brains use timing and synchronization in the way that they compute, but this is largely ignored in modern neural nets. We believe neural timing is key for the flexibility and adaptability of biological intelligence.
We propose a new neural architecture, “Continuous Thought Machines” (CTMs), which is built from the ground up to use neural dynamics as a core representation for intelligence. By using neural dynamics as a first-class representational citizen, CTMs naturally perform adaptive computation.
Many emergent, interesting behaviors arise as a result: CTMs solve mazes by observing a raw maze image and producing step-by-step instructions directly from its neural dynamics. When tasked with image recognition, the CTM naturally takes multiple steps to examine different parts of the image before making its decision. This step-by-step approach not only makes its behavior more interpretable but also improves accuracy: the longer it “thinks,” the more accurate its answers become.
We also found that this allows the CTM to decide to spend less time thinking on simpler images, thus saving energy. When identifying a gorilla, for example, the CTM’s attention moves from eyes to nose to mouth in a pattern remarkably similar to human visual attention.
I think this work underscores an important, yet often lost, synergy between neuroscience and AI. While modern AI is ostensibly brain-inspired, the two fields often operate in surprising isolation. By starting with such inspiration and iteratively following the emergent, interesting behaviors, we developed a model with unexpected capabilities, such as its surprisingly strong calibration in classification tasks, a feature that was not explicitly designed for.
When we initially asked, “why do this research?”, we hoped the journey of the CTM would provide compelling answers. By embracing light biological inspiration and pursuing the novel behaviors observed, we have arrived at a model with emergent capabilities that exceeded our initial designs. We are committed to continuing this exploration, borrowing further concepts to discover what new and exciting behaviors will emerge, pushing the boundaries of what AI can achieve. -
Kenneth Stanley reposted
Lux Capital
@Lux_Capital
1/ We're dedicating $100M to support American scientists facing funding cuts through the Lux Science Helpline, an expansion of our Lux Labs program. To compete in the future, innovation must be funded 🧪💡
-
Kenneth Stanley
@kenneth0stanley
The objective paradox in action.
Jack Morris
@jxmnop
it's interesting to see the big AI labs (at least OpenAI, anthropic, google, xai?) converge on EXACTLY the same extremely specific list of products:
- a multimodal chatbot
- with a long-compute 'reasoning' mode
- and something like "deep research"
reminds me of a few years ago, when instagram tiktok youtube all converged to ~the same app
why does this happen? -
Kenneth Stanley
@kenneth0stanley
Really cool and original upcoming exhibition of CPPNs (compositional pattern producing networks) here!
Cédric
@cedcolas
i just got an art grant from the council for the arts at MIT!
*Tangible Dreams* will let visitors experiment and play with a physical neural network generating images real-time—by twisting knobs and switches, by reconnecting nodes together @ArtsatMIT -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Automatic Capability Discovery is another.
x.com/jeffclune/status/1889568685632667672
arxiv.org/abs/2502.07577
Jeff Clune
@jeffclune
Introducing Automated Capability Discovery!
ACD automatically identifies surprising new capabilities and failure modes in foundation models, via "self-exploration" (models exploring their own abilities).
Led by @cong_ml & @shengranhu 🔬🤖🧠🔎 [1/9] -
Kenneth Stanley
@kenneth0stanley
Completely agree with this point. How can you really take AI safety seriously without grappling directly and explicitly with the fact the world is open-ended? Rainbow Teaming is a great example!
Mikayel Samvelyan
@_samvelyan
Incredible keynote by @_rockt! Some might fear that open-endedness is a disease for AI safety; I say it might be the remedy.
Our 🌈Rainbow Teaming work proved these methods are great at finding & fixing LLM vulnerabilities. Hope the safety folks use more of these powerful methods -
Kenneth Stanley reposted
-
Kenneth Stanley reposted
Edward Hughes
@edwardfhughes
2025 is the year of open-endedness.
-
Kenneth Stanley reposted
Andrew Beam
@AndrewLBeam
So excited that we get to work with @kenneth0stanley on open-endedness for scientific AI @LilaSciences!
Ken is truly one of the most original thinkers I've ever met in addition to being an amazing colleague.
PS - His open-endedness team is hiring, come join us!
Job links 👇 -
Kenneth Stanley reposted
-
Kenneth Stanley
@kenneth0stanley
Awesome to see a keynote on open-endedness at #iclr - way to go @_rockt ! You have the right message at the right time and I appreciate the callout in the abstract. I wish I was there to see this. Open-endedness is the next frontier for AI as the benchmark race loses its allure.
Jeff Clune
@jeffclune
Very excited for this keynote by @_rockt! Awesome to see open-endedness go from a niche (😉) area to a keynote at #iclr ! 🌱🌿🌳🌲🍀🌍✨ 📈 🧬🧪 cc @joelbot3000 @kenneth0stanley
-
Kenneth Stanley reposted
Paras Chopra
@paraschopra
My book is out for pre-orders! 🥳🎉
First 100 preorders get a signed copy from me.
Grab your copy 👇 -
Kenneth Stanley
@kenneth0stanley
This would definitely be an unplanned outcome for a career researching AI!
Schrodinger's Cockatoo
@MaxDiffusionRL
I really want to create a taoist temple and enshrine @kenneth0stanley as a Daoist God.
-
Kenneth Stanley
@kenneth0stanley
Congrats to @kanjun and @imbue_ai on a very creative angle on AI-assisted coding! And by the way, this is what happens when a company is unafraid of open-ended exploration, which I happen to know Kanjun has artfully incorporated into Imbue's DNA. More great stuff sure to come!
Kanjun 🐙
@kanjun
Today, AI can generate tons of code—but how do we know if it's good?
That's why we built Sculptor: the first coding agent environment.
Sculptor helps you catch issues, write tests, and improve your code—all while you work in your favorite editor. -
Kenneth Stanley reposted
hardmaru
@hardmaru
Excited to release our technical report: “The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search”‼️
pub.sakana.ai/ai-scientist-v2/paper/
The AI Scientist-v2 incorporates an “Agentic Tree Search” approach into the workflow, enabling deeper and more systematic exploration of scientific hypotheses by harnessing principles from the field of open-endedness.
Super proud of the amazing progress our team is making❗
Sakana AI
@SakanaAILabs
Introducing The AI Scientist-v2, which produced the 1st fully AI-generated paper to pass peer review at a workshop level (at @iclr_conf 2025)!
Tech Report: pub.sakana.ai/ai-scientist-v2/paper/
GitHub: github.com/SakanaAI/AI-Scientist-v2
V2 adds:
🧠 Agentic Tree Search (open-ended idea exploration)
💡 More Generality (reduced template reliance)
👀 VLM Reviewer (better figures and understanding)
⚡ Parallel Execution
By @_yutaroyamada @RobertTLange @cong_ml @shengranhu @_chris_lu_ @j_foerst @jeffclune @hardmaru (Sakana AI, UBC, Oxford) -
Kenneth Stanley reposted
Sakana AI
@SakanaAILabs
Introducing The AI Scientist-v2, which produced the 1st fully AI-generated paper to pass peer review at a workshop level (at @iclr_conf 2025)!
Tech Report: pub.sakana.ai/ai-scientist-v2/paper/
GitHub: github.com/SakanaAI/AI-Scientist-v2
V2 adds:
🧠 Agentic Tree Search (open-ended idea exploration)
💡 More Generality (reduced template reliance)
👀 VLM Reviewer (better figures and understanding)
⚡ Parallel Execution
By @_yutaroyamada @RobertTLange @cong_ml @shengranhu @_chris_lu_ @j_foerst @jeffclune @hardmaru (Sakana AI, UBC, Oxford) -
Kenneth Stanley reposted
Emmett Shear
@eshear
Current machine learning is all about mastering convergence to an objective within as wide a domain as possible.
The missing piece is mastering divergence. -
Kenneth Stanley reposted
Maxence Faldor
@maxencefaldor
How can we collect the best stepping stones for open-ended discovery?
I am excited to share Learned Quality-Diversity — a family of algorithms meta-optimized to be efficient stepping stone collectors!
📄: arxiv.org/abs/2502.02190
🌟: github.com/maxencefaldor/learned-qd
work done in collaboration with @RobertTLange from @SakanaAILabs and @CullyAntoine. -
Kenneth Stanley reposted
Emmett Shear
@eshear
As you might have guessed from following my posts here, I've been thinking and working on questions around alignment and learning and agency for a while, particularly for digital systems.
Excited to share the work we're doing at Softmax publicly in the future! -
Kenneth Stanley reposted
Julian Togelius
@togelius
Remember our work on Word2World, where we generate playable worlds based on stories? Continuing this line of research, we have been working on generating Minecraft environments that tell playable stories. Check it out 👇
Shuo Huang
@ShuoHuang131026
🚀Excited to share our new research: Word2Minecraft (arxiv.org/abs/2503.16536), which is an improvement to Word2World (arxiv.org/abs/2405.06686). Thanks to my supervisors Prof. @togelius, Dr. Steven James and Muhammad @utheprodigyn for their guidance and support! 1/13
-
Kenneth Stanley reposted
Joel Lehman
@joelbot3000
Grateful to be included in @cosmos_inst's push to bridge AI+philosophy. Excited by the possibility of ML models that embody flavors of virtue, love, enlightenment appropriate for machines -- which requires reimagining philosophical concepts to fit+extend formalisms of ML
Brendan McCord 🏛️ x 🤖
@Brendan_McCord
📣 Announcing the newest cohort of Cosmos Fellows 🙌
First up: Joel Lehman @joelbot3000!
Joel is an ML researcher who:
🔹 Co-led the Open-Endedness Team at @OpenAI
🔹 Was a founding member of @UberAILabs
🔹 Co-authored the book "Why Greatness Cannot Be Planned"
His fellowship will explore "machine virtue"--reworking philosophical conceptions of virtue to be fittingly embodied by ML systems so they can serve human flourishing.
This exciting work bridges ML formalisms with philosophy and positive psychology.
"The philosophy-to-code pipeline is exactly what I want to help develop at Cosmos." (more on that in the final tweet.) -
Kenneth Stanley reposted
Paul Millerd
@p_millerd
just started this book
and to the 10 people that recommended it to me - they were right
very resonant with how I approach life and vibes very well with my two books -
Kenneth Stanley reposted
Tim Ferriss
@tferriss
Ev Williams (@ev) on "Why Greatness Cannot Be Planned: The Myth of the Objective"
This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
Congrats @joelbot3000 ! They chose the right person for this fellowship!
Brendan McCord 🏛️ x 🤖
@Brendan_McCord
📣 Announcing the newest cohort of Cosmos Fellows 🙌
First up: Joel Lehman @joelbot3000!
Joel is an ML researcher who:
🔹 Co-led the Open-Endedness Team at @OpenAI
🔹 Was a founding member of @UberAILabs
🔹 Co-authored the book "Why Greatness Cannot Be Planned"
His fellowship will explore "machine virtue"--reworking philosophical conceptions of virtue to be fittingly embodied by ML systems so they can serve human flourishing.
This exciting work bridges ML formalisms with philosophy and positive psychology.
"The philosophy-to-code pipeline is exactly what I want to help develop at Cosmos." (more on that in the final tweet.) -
Kenneth Stanley
@kenneth0stanley
An interesting challenge for “agentic AI” is that people actually like having agency. Not in all situations, and not all the time, but marketing and human nature will clash here in unpredictable ways as efficiency competes against the joy of self-determination.
-
Kenneth Stanley reposted
Anne-Laure Le Cunff
@neuranne
I want to create an expanded reading list of books that pair well with Tiny Experiments. Books like:
• Wanting @lukeburgis
• The Pathless Path @p_millerd
• Why Greatness Cannot Be Planned @kenneth0stanley
Any suggestions to add to the list? -
Kenneth Stanley
@kenneth0stanley
More of this kind of exploration is needed. Nice work @_joelsimon !
Joel Simon
@_joelsimon
New research project: Lluminate - an evolutionary algorithm that helps LLMs break free from generating predictable, similar outputs. Combining evolutionary principles with creative thinking strategies can illuminate the space of possibilities.
www.joelsimon.net/lluminate -
Kenneth Stanley reposted
Joel Simon
@_joelsimon
New research project: Lluminate - an evolutionary algorithm that helps LLMs break free from generating predictable, similar outputs. Combining evolutionary principles with creative thinking strategies can illuminate the space of possibilities.
www.joelsimon.net/lluminate -
Kenneth Stanley reposted
hardmaru
@hardmaru
This was a fun experiment we ran while developing The AI Scientist-v2. With the permission of ICLR, we submitted an AI-generated paper to an ICLR workshop that passed the peer-review process.
We documented the entire process and our learnings in a blog: sakana.ai/ai-scientist-first-publication/
As AI researchers, we also wrote our own (human) reviews documenting our own assessment and critiques of the AI-generated papers, and conducted code reviews on the computational experiments conceived by The AI Scientist-v2, which you might find interesting!
The AI-generated papers and our analysis of them are also published on our GitHub: github.com/SakanaAI/AI-Scientist-ICLR2025-Workshop-Experi…
As we embrace artificial novelty search and open-ended discovery with AI, I believe computational creativity can enable frontier LLMs to produce even more novel and imaginative ideas (and if these ideas are related to AI / ML, can be tested with actual computational experiments conceived by AI). Perhaps one day, AI systems can produce groundbreaking scientific discoveries (or maybe, an accepted NeurIPS or ICLR paper 😛)
Sakana AI
@SakanaAILabs
The AI Scientist Generates its First Peer-Reviewed Scientific Publication
We’re proud to announce that a paper produced by The AI Scientist-v2 passed the peer-review process at a workshop in ICLR, a top AI conference.
Read more about this experiment → sakana.ai/ai-scientist-first-publication/
To our knowledge, this is the first fully AI-generated paper that has passed the same peer-review process that human researchers go through. The paper was produced by an improved version of the original AI Scientist, called The AI Scientist-v2. We’ll be sharing the full details of v2 in an upcoming release.
We conducted this experiment with the full cooperation of both the ICLR leadership and the organizers of the ICLR workshop, @ICBINBWorkshop. We (@_yutaroyamada @cong_ml @shengranhu @RobertTLange) proudly collaborated with UBC (@jeffclune) and Oxford (@FLAIR_Ox) on this exciting project. -
Kenneth Stanley reposted
Akarsh Kumar
@akarshkumar0101
I recently gave a talk at the MIT Embodied Intelligence Seminar on Automating the Search for Artificial Life with Foundation Models, check it out!
www.youtube.com/watch?v=S6uLxSFli-E -
Kenneth Stanley reposted
Machine Learning Street Talk
@MLStreetTalk
This might be the job opportunity of a lifetime to work with one of my biggest heroes in AI - Kenneth Stanley! Do get in touch with him if you are interested - looks incredible.
Kenneth Stanley
@kenneth0stanley
Important announcement (with job opportunities!): I’m thrilled to share that I just joined @LilaSciences as SVP of Open-Endedness! Lila is a new name in the AI space, but one you will be hearing a lot from. Their unique mission to pursue Scientific Superintelligence could not align better with my interest in open-ended creativity. Lila is about the entire scientific enterprise, not just a single constrained domain like drug discovery.
And of course, the history of scientific progress is an unmistakable precedent for open-ended discovery and Why Greatness Cannot Be Planned. For AI itself to join this epic quest beyond just a supporting role, to create new magic that the human mind has yet to imagine, we will need to take creativity seriously well beyond the conventional pillars of more data, more compute, and more time on inference. That may be where the frontier labs currently are, but if you know about open-endedness, you know that open-ended creativity is not about taking a test and getting a good score.
Lila understands the difference and is willing to invest to help me create the best open-endedness team in the world (as part of their overall AI effort) to make it happen. So I want to invite you, if you think this opportunity sounds as exciting as I do, to reach out if you’re interested in joining my team. I’m looking for a diverse range of expertise: pre-training, fine-tuning, RLHF, distillation, mechanistic interpretability, and yes - quality diversity techniques!
We’re going to do things on my team that nobody else is doing. This will not be the usual roadmap. Compensation will be industry competitive and our team will be based in San Francisco with a hybrid work schedule. 1/n -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Your chance to work with one of the all-time great minds in science! What stepping stones will this team create and add to the archive of human knowledge? Exciting times! 🚀🚀🚀🚀🚀🚀
Kenneth Stanley
@kenneth0stanley
Important announcement (with job opportunities!): I’m thrilled to share that I just joined @LilaSciences as SVP of Open-Endedness! Lila is a new name in the AI space, but one you will be hearing a lot from. Their unique mission to pursue Scientific Superintelligence could not align better with my interest in open-ended creativity. Lila is about the entire scientific enterprise, not just a single constrained domain like drug discovery.
And of course, the history of scientific progress is an unmistakable precedent for open-ended discovery and Why Greatness Cannot Be Planned. For AI itself to join this epic quest beyond just a supporting role, to create new magic that the human mind has yet to imagine, we will need to take creativity seriously well beyond the conventional pillars of more data, more compute, and more time on inference. That may be where the frontier labs currently are, but if you know about open-endedness, you know that open-ended creativity is not about taking a test and getting a good score.
Lila understands the difference and is willing to invest to help me create the best open-endedness team in the world (as part of their overall AI effort) to make it happen. So I want to invite you, if you think this opportunity sounds as exciting as I do, to reach out if you’re interested in joining my team. I’m looking for a diverse range of expertise: pre-training, fine-tuning, RLHF, distillation, mechanistic interpretability, and yes - quality diversity techniques!
We’re going to do things on my team that nobody else is doing. This will not be the usual roadmap. Compensation will be industry competitive and our team will be based in San Francisco with a hybrid work schedule. 1/n -
Kenneth Stanley
@kenneth0stanley
Important announcement (with job opportunities!): I’m thrilled to share that I just joined @LilaSciences as SVP of Open-Endedness! Lila is a new name in the AI space, but one you will be hearing a lot from. Their unique mission to pursue Scientific Superintelligence could not align better with my interest in open-ended creativity. Lila is about the entire scientific enterprise, not just a single constrained domain like drug discovery.
And of course, the history of scientific progress is an unmistakable precedent for open-ended discovery and Why Greatness Cannot Be Planned. For AI itself to join this epic quest beyond just a supporting role, to create new magic that the human mind has yet to imagine, we will need to take creativity seriously well beyond the conventional pillars of more data, more compute, and more time on inference. That may be where the frontier labs currently are, but if you know about open-endedness, you know that open-ended creativity is not about taking a test and getting a good score.
Lila understands the difference and is willing to invest to help me create the best open-endedness team in the world (as part of their overall AI effort) to make it happen. So I want to invite you, if you think this opportunity sounds as exciting as I do, to reach out if you’re interested in joining my team. I’m looking for a diverse range of expertise: pre-training, fine-tuning, RLHF, distillation, mechanistic interpretability, and yes - quality diversity techniques!
We’re going to do things on my team that nobody else is doing. This will not be the usual roadmap. Compensation will be industry competitive and our team will be based in San Francisco with a hybrid work schedule. 1/n
Kenneth Stanley
@kenneth0stanley
Job Postings:
For Research Scientist: job-boards.greenhouse.io/lila/jobs/7890006002
For Engineer: job-boards.greenhouse.io/lila/jobs/7890007002
Kenneth Stanley
@kenneth0stanley
Learn more about Lila: www.lila.ai
Lila Blog Post: www.lila.ai/news/join-our-mission -
Kenneth Stanley
@kenneth0stanley
It’s become popular these days in AI to celebrate that we now have “two dimensions of scaling!” But having more than one dimension only highlights that intelligence is multidimensional and the number of dimensions is not necessarily only 2 either.
-
Kenneth Stanley
@kenneth0stanley
This is a really nice argument for the gap between objective benchmark performance and open-endedness. It’s worth thinking about the region of achievement where conventional intelligence and creative genius diverge.
Thomas Wolf
@Thom_Wolf
I shared a controversial take the other day at an event and I decided to write it down in a longer format: I’m afraid AI won't give us a "compressed 21st century".
The "compressed 21st century" comes from Dario's "Machine of Loving Grace" and if you haven’t read it, you probably should, it’s a noteworthy essay. In a nutshell the paper claims that, over a year or two, we’ll have a "country of Einsteins sitting in a data center”, and it will result in a compressed 21st century during which all the scientific discoveries of the 21st century will happen in the span of only 5-10 years.
I read this essay twice. The first time I was totally amazed: AI will change everything in science in 5 years, I thought! A few days later I came back to it and, re-reading it, I realized that much of it seemed like wishful thinking at best.
What we'll actually get, in my opinion, is “a country of yes-men on servers” (if we just continue on current trends). Let me explain the difference with a small part of my personal story.
I’ve always been a straight-A student. Coming from a small village, I joined the top French engineering school before getting accepted to MIT for PhD. School was always quite easy for me. I could just get where the professor was going, where the exam's creators were taking us and could predict the test questions beforehand.
That’s why, when I eventually became a researcher (more specifically a PhD student), I was completely shocked to discover that I was a pretty average, underwhelming, mediocre researcher. While many colleagues around me had interesting ideas, I was constantly hitting a wall. If something was not written in a book I could not invent it unless it was a rather useless variation of a known theory. More annoyingly, I found it very hard to challenge the status-quo, to question what I had learned. I was no Einstein, I was just very good at school. Or maybe even: I was no Einstein in part *because* I was good at school.
History is filled with geniuses struggling during their studies. Edison was called "addled" by his teacher. Barbara McClintock got criticized for "weird thinking" before winning a Nobel Prize. Einstein failed his first attempt at the ETH Zurich entrance exam. And the list goes on.
The main mistake people usually make is thinking Newton or Einstein were just scaled-up good students, that a genius comes to life when you linearly extrapolate a top-10% student.
This perspective misses the most crucial aspect of science: the skill to ask the right questions and to challenge even what one has learned. A real science breakthrough is Copernicus proposing, against all the knowledge of his days -in ML terms we would say “despite all his training dataset”-, that the earth may orbit the sun rather than the other way around.
To create an Einstein in a data center, we don't just need a system that knows all the answers, but rather one that can ask questions nobody else has thought of or dared to ask. One that writes 'What if everyone is wrong about this?' when all textbooks, experts, and common knowledge suggest otherwise.
Just consider the crazy paradigm shift of special relativity and the guts it took to formulate a first axiom like “let’s assume the speed of light is constant in all frames of reference” defying the common sense of these days (and even of today…)
Or take CRISPR, generally considered to be an adaptive bacterial immune system since the 80s until, 25 years after its discovery, Jennifer Doudna and Emmanuelle Charpentier proposed to use it for something much broader and general: gene editing, leading to a Nobel prize. This type of realization –"we've known XX does YY for years, but what if we've been wrong about it all along? Or what if we could apply it to the entirely different concept of ZZ instead?” is an example of out-side-of-knowledge thinking –or paradigm shift– which is essentially making the progress of science.
Such paradigm shifts happen rarely, maybe 1-2 times a year and are usually awarded Nobel prizes once everybody has taken stock of the impact. However rare they are, I agree with Dario in saying that they take the lion’s share in defining scientific progress over a given century while the rest is mostly noise.
Now let’s consider what we’re currently using to benchmark recent AI model intelligence improvement. Some of the most recent AI tests are for instance the grandiosely named "Humanity's Last Exam" or "Frontier Math". They consist of very difficult questions –usually written by PhDs– but with clear, closed-end, answers.
These are exactly the kinds of exams where I excelled in my field. These benchmarks test if AI models can find the right answers to a set of questions we already know the answer to.
However, real scientific breakthroughs will come not from answering known questions, but from asking challenging new questions and questioning common conceptions and previous ideas.
Remember Douglas Adams' Hitchhiker's Guide? The answer is apparently 42, but nobody knows the right question. That's research in a nutshell.
In my opinion this is one of the reasons LLMs, while they already have all of humanity's knowledge in memory, haven't generated any new knowledge by connecting previously unrelated facts. They're mostly doing "manifold filling" at the moment - filling in the interpolation gaps between what humans already know, somehow treating knowledge as an intangible fabric of reality.
We're currently building very obedient students, not revolutionaries. This is perfect for today’s main goal in the field of creating great assistants and overly compliant helpers. But until we find a way to incentivize them to question their knowledge and propose ideas that potentially go against past training data, they won't give us scientific revolutions yet.
If we want scientific breakthroughs, we should probably explore how we’re currently measuring the performance of AI models and move to a measure of knowledge and reasoning able to test if scientific AI models can for instance:
- Challenge their own training data knowledge
- Take bold counterfactual approaches
- Make general proposals based on tiny hints
- Ask non-obvious questions that lead to new research paths
We don't need an A+ student who can answer every question with general knowledge. We need a B student who sees and questions what everyone else missed.
---
PS: You might be wondering what such a benchmark could look like. Evaluating it could involve testing a model on some recent discovery it should not know yet (a modern equivalent of special relativity) and explore how the model might start asking the right questions on a topic it has no exposure to the answers or conceptual framework of. This is challenging because most models are trained on virtually all human knowledge available today but it seems essential if we want to benchmark these behaviors. Overall this is really an open question and I’ll be happy to hear your insightful thoughts. -
Kenneth Stanley reposted
Aryan Mangla
@Aryan_Mangla_
Week 9 of 50
“NeuroEvolution of Augmenting Topologies(NEAT)”
Traditional methods use fixed neural networks, but what if the AI could build its own brain(neural) structure as it learns? That’s the big idea behind NEAT
Full breakdown 👇
Authors - @kenneth0stanley, Risto M. -
Kenneth Stanley reposted
Alex Levy
@alex_levox
well, I kinda did!!!
wrote this book for my 18 year old self, and discuss @kenneth0stanley's ideas (what AI teaches us about greatness)
a.co/d/icdlj9G -
Kenneth Stanley reposted
hardmaru
@hardmaru
“Evolution and The Knightian Blindspot of Machine Learning”
arxiv.org/abs/2501.13075
Fascinating paper. Makes the case that the harsh filter of evolution enabled organisms (including us) to navigate unexpected events (“unknown unknowns”), something that AI systems struggle with.
Joel Lehman
@joelbot3000
new paper: "Evolution and the Knightian Blindspot of Machine Learning"
Our ever-changing world bubbles with surprise and complexity. General AI must include handling unforeseen situations with grace. Yet this issue largely lies outside AI's formalisms: a blind spot. (1/n) -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
We're excited to announce the first Evolving Self-organisation workshop at @GeccoConf!
Submission deadline: March 26, 2025
More information: evolving-self-organisation-workshop.github.io -
Kenneth Stanley reposted
Nikolaj Astrup
@nikolaj_astrup
Someone should write a combination of these two books, but targeted high schoolers / young adults, who are just about to choose their path.
-
Kenneth Stanley reposted
MindBranches
@MindBranches
Book Summary: “Why Greatness Cannot Be Planned”
-
Kenneth Stanley
@kenneth0stanley
So many philosophical implications to progress in AI and so few venues really addressing them head on. My friend @tobias_rees has created the very unique @limn_ai to fill the gap!
Tobias Rees
@tobias_rees
we live in turbulent, wild times.
times that need philosophers in the wild 🎈
philosophers who immerse themselves in AI and render visible its philosophical stakes
Let me introduce you to @limn_aiThis media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Tom Butler-Bowdon
@tombutlerbowdon
🧵 The Counterintuitive Truth About Success Most People Miss:
Chasing Goals Directly Often Fails
1/ Ever been told to set goals and pursue them relentlessly? What if that advice is fundamentally wrong?
2/ Economist @ProfJohnKay discovered something fascinating: The DIRECT path to success is often the WORST way to achieve your objectives. He calls this principle "obliquity" 🧭
3/ Take @Boeing: They didn't become aviation legends by obsessing over profits. They succeeded by LOVING planes. Profit was the side effect, not the target.
(A few days ago Boeing posted its results for 2024: losses of $11.8 billion and revenue down 14%, with significantly fewer deliveries of new aircraft in a time when aviation is expanding.)
4/ Same with @Merck Pharmaceuticals. CEO George Merck said: "Medicine is for the people. The profits follow." When they forgot this, they started making catastrophic mistakes like Vioxx.
5/ Here's the mind-blowing truth: Great achievements often come through SIDEWAYS exploration, not straight-line planning.
This is what computer scientists @kenneth0stanley and @joelbot3000 discovered and say in their amazing book "Greatness Cannot Be Planned"
6/ Rock 'n' roll wasn't a strategic plan. It happened when #elvis "played the fool", mixing jazz, blues, gospel, and country in a revolutionary blend.
7/ #bitcoin didn't emerge from a "blockchain goal" - it combined cryptography, bookkeeping, and decentralization in a totally unexpected way.
8/ The most successful people aren't goal-setters. They're CURIOUS EXPLORERS who follow fascinating "stepping stones" (Stanley & Lehman) without knowing the final destination.
9/ Nature doesn't have goals. Evolution creates complexity through endless, seemingly random combinations. Your creativity works the same way.
10/ Stop asking "What's my five-year plan?" Start asking: "What unexpected connections can I explore today?"
11/ Your unexpected passion might be your greatest asset. Allow yourself to wander. Allow yourself to get lost. Breakthrough awaits in the unexplored. 🚀
#success #innovation #personalgrowth
Full article:
tombutlerbowdon.substack.com/p/the-hidden-cost-of-goals -
Kenneth Stanley reposted
Aaron Stupple
@astupple
Interestingness is treated a frivolity, a side project. But it is precious.
-
Kenneth Stanley reposted
Sakana AI
@SakanaAILabs
We’re pleased to announce that our paper, “Evolutionary Optimization of Model Merging Recipes,” has been accepted to Nature Machine Intelligence and published today! The latest version includes new experimental results that further validate our approach.
www.nature.com/articles/s42256-024-00975-8
Back in March 2024, Sakana AI released “Evolutionary Model Merge,” which generated some attention in the AI community. It has since been implemented in well-known open-source frameworks like mergekit and Optuna Hub, enabling diverse users to create and publish many unique models. Multiple internal and external teams have been pursuing follow-up research; one example is “CycleQD,” which was accepted to ICLR 2025. We are happy to incorporate all the feedback from the community and from our reviewers, and release additional results into our Nature Machine Intelligence publication.
We view “Evolutionary Model Merge” as the first step in our long-term R&D efforts. Sakana AI will continue to learn from the principles of nature and forge new frontiers in foundational model development. Stay tuned! -
Kenneth Stanley
@kenneth0stanley
DeepSeek has triggered some heated geopolitical and market competition discussion, but the deeper research question is, does being able to train something for 20x less cost mean that you can get something 20x smarter for the original cost? If not, what’s the scaling implication?
-
Kenneth Stanley reposted
Tarin Ziyaee
@tarinziyaee
Remember Moravec's paradox?
"Corner cases".
"Long tails".
You can play whack-a-mole with them.
Or, you can confront their root causes directly.
Our new paper "Evolution and The Knightian Blindspot of Machine Learning" argues we should do the latter. -
Kenneth Stanley reposted
Joel Lehman
@joelbot3000
new paper: "Evolution and the Knightian Blindspot of Machine Learning"
Our ever-changing world bubbles with surprise and complexity. General AI must include handling unforeseen situations with grace. Yet this issue largely lies outside AI's formalisms: a blind spot. (1/n) -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Research Idea: Predict abstract properties of text in addition to next-word prediction as helpful auxiliary losses. Has this been tried?
E.g. Generate a summary based on the entire text and then predict that summary at different points in the text. For example, summarize a murder mystery and then predict that summary 1/10th, 1/3rd, 1/2, 7/8ths, etc. of the way through (so you have to predict the outcome). These predictions are in a semantic abstract space vs. word by word, and would be done in addition to next-word prediction.
For videos, this involves predicting what will happen abstractly or semantically, rather than focusing on specific (and often impossible-to-predict) pixels—a concept that @ylecun has advocated for years.
You can imagine many other semantic losses (invent good questions about what will happen and answer them, etc.).
Has someone done this? If so, did it work? If not, it would be a fascinating area to explore!" -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Our UBC lab + collaborators had 3/3 papers accepted to #iclr2025!
🚀🚀🚀🍻🍻🍻💥💥💥👊👊👊💡💡💡
Thanks @iclr_conf ACs, reviewers, and organizers. 🙏
See you in Singapore! 🇸🇬 -
Kenneth Stanley reposted
Tim Rocktäschel
@_rockt
Recurring question: So what? It can't be used to train Deep NNs, let alone LLMs.
That's true. ES doesn't work well in high dimensional spaces. As it maintains a population, memory requirements (if you are using ES to train Deep NNs) are prohibitive and sample efficiency is not competitive with Adam. Moreover, Choromanska et al. (2015). The Loss Surfaces of Multilayer Networks. arXiv. doi.org/10.48550/arXiv.1412.0233 shows that as dimensionality increases, most local minima are "good" in the sense that they generalize well and are close in value to the global minimum. So most of the time we don't care about finding the global optimum.
That said, many successful recent in-context learning papers can be understood under the lens of ES, for example, Wang et al. (2023). Voyager: An Open-Ended Embodied Agent with Large Language Models. arxiv.org/abs/2305.16291, FunSearch by Romera-Paredes et al. (2024). Mathematical discoveries from program search with large language models. Nature, 625(7995), 468–475. doi.org/10.1038/s41586-023-06924-6, Fernando et al. (2024). Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution. ICML. arxiv.org/abs/2309.16797), Samvelyan et al. (2024). Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts. NeurIPS. doi.org/10.48550/arXiv.2402.16822), Zhang et al. (2023). OMNI: Open-endedness via Models of human Notions of Interestingness. ICML. arxiv.org/abs/2306.01711, Faldor et al. (2024). OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code. arXiv. doi.org/10.48550/arXiv.2405.15568, Lu et al. (2024). The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery. arXiv. doi.org/10.48550/arXiv.2408.06292). All of these works use foundation models to either vary or select artifacts (or both). All of them implement an evolutionary strategy in some way, but not on the NN parameter level. Most of them keep an archive of novel and interesting stepping stones that become the basis for more successful solutions later.
Tim Rocktäschel
@_rockt
Highly non-convex toy optimization problem.
SGD vs Adam vs Evolutionary Strategies (ES).
Let's gooooooo... -
Kenneth Stanley reposted
gabe
@gabekahen_
Gifting a buddy of mine my beloved copy of the enlightening *Why Greatness Cannot Be Planned* by my GOATs @kenneth0stanley and @joelbot3000, and it kinda feels like divorce. This is a must read for literally everyone—it totally reshaped how I think about innovation and creativity
-
Kenneth Stanley
@kenneth0stanley
There’s a lot of love right now for System 2 thinking in AI. But imagine if System 2 is overrated? It would be a classic deceptive trap. How could it be overrated? Well because if it becomes easier to make progress on it than on System 1, human nature will yearn to embrace it.
-
Kenneth Stanley reposted
Sakana AI
@SakanaAILabs
We’re excited to introduce Transformer², a machine learning system that dynamically adjusts its weights for various tasks!
sakana.ai/transformer-squared
Adaptation is a remarkable natural phenomenon, like how the octopus can blend in with its environment, or how the brain rewires itself after injury. We believe our new system paves the way for a new generation of adaptive AI models, modifying their own weights and architecture to adapt to the nature of the tasks they encounter, embodying living intelligence capable of continuous change and lifelong learning.
Paper: arxiv.org/abs/2501.06252
GitHub: github.com/SakanaAI/self-adaptive-llms -
Kenneth Stanley
@kenneth0stanley
If a cutting-edge conversational LLM was trained from scratch but withholding from all phases of training any information about what it is or where it came from, I wonder, what would it say about itself when asked about its identity or origins?
-
Kenneth Stanley
@kenneth0stanley
There is something fascinating about the idea of video game worlds where agents lead rich lives with natural language interactions with each other, but the on-device cost of LLMs makes it a challenge! What my friends at @StudioAtelico created can blow this whole opportunity open!
Piero Molino
@w4nderlus7
Today, I’m excited to unveil a project that’s incredibly close to my heart. As a lifelong gamer, I’ve always dreamed of pushing the boundaries of what’s possible in video games using my expertise in AI. Over the past year, my team at @StudioAtelico and I have been blending the magic of gaming with AI to create something special and groundbreaking: an on-device AI engine for games that empowers NPCs, agents, and natural language interactions in ways that were previously impossible or prohibitively expensive.
To showcase its potential, we built GARP, a tech demo inspired by Generative Agents. GARP demonstrates how our AI engine runs in real time, locally on-device—all without relying on the cloud and for free.
Can’t wait to see what game developers will build with the Atelico AI Engine! 🚀
Details and links below 👇This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
I took the plunge and engaged @liron on his show, Doom Debates! Unlike some, my aim was not to impose a prognostication on my expectation of doom, but rather to argue strenuously that open-endedness needs to be thrown into the mix when discussing this topic, because any future AGI or superintelligence will inevitably be open-ended, so we must learn to grapple with that fact. Awareness of open-endedness is regrettably low on all sides among prognosticators on our AI future, and it deserves a much bigger seat at the table. Thank you to @liron (who is not afraid to push back!) for giving me this opportunity and for providing the forum to make the argument.
Liron Shapira
@liron
📣 My AI doom debate with @Kenneth0Stanley, former Research Team Lead at @OpenAI
Prof. Kenneth Stanley is a former Research Science Manager at OpenAI leading the Open-Endedness Team in 2020-2022.
His book, Why Greatness Cannot Be Planned: The Myth of the Objective, argues that creating objectives is actually COUNTERPRODUCTIVE for intelligent agents.
He was previously a Professor of Computer Science at the University of Central Florida, and the head of Core AI Research at Uber.
Ken and I debate whether superintelligent AI will or won't be guided by goals, and compare our views on AI doom.
Topics covered:
* Ken's role at OpenAI
* Ken's concepts of Open-Endedness and Divergence
* Open-Endedness of Evolution
* The concept of an Optimization Process
* What's Your P(Doom)™
* Instrumental Convergence
* Mitigating AI Risks -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
I greatly enjoyed this interview on @MLStreetTalk It covers a wide range of topics I am interested in (open-endedness, AI-GAs, AI safety, etc.), includes my scientific origin story, and has great visuals edited in. Thanks Tim Scarfe!
www.youtube.com/watch?v=mw5WIDGRLnA -
Kenneth Stanley reposted
Kenneth Stanley
@kenneth0stanley
I think my former OpenAI colleague @nickcammarata makes a good point here that is very aligned with arguments from open-endedness, though said in his own unique way. I’d add that I think Nick is largely pointing to the perennial benchmark obsession that dominates the field. And this obsession is very related to the objective paradox we describe in Why Greatness Cannot Be Planned, which is the problem that the more you fixate on measurable progress towards a goal, the more likely you are heading into a brick wall and the less able you are to innovate. We should remember that the reason ChatGPT was one of the most quickly adopted technologies in history was not because people were wowed by some benchmark results, but rather because the qualitative experience was interesting, even fascinating! So for those itching to break out of the objective paradox, instead of thinking about how to ace a test (as tempting as that is), try to imagine what experience would be similarly fascinating to the general public without needing any benchmark result at all. That is the path of the interesting instead of the path of the objective, and indeed we often collapse to something like only 1% of effort given to the path of the interesting. Those 99% who are trapped wallowing in false security blanket of the objective paradox dwell there at their peril.
-
Post not available here, view on XPost not available here, view on X
Kenneth Stanley
@kenneth0stanley
I think my former OpenAI colleague @nickcammarata makes a good point here that is very aligned with arguments from open-endedness, though said in his own unique way. I’d add that I think Nick is largely pointing to the perennial benchmark obsession that dominates the field. And this obsession is very related to the objective paradox we describe in Why Greatness Cannot Be Planned, which is the problem that the more you fixate on measurable progress towards a goal, the more likely you are heading into a brick wall and the less able you are to innovate. We should remember that the reason ChatGPT was one of the most quickly adopted technologies in history was not because people were wowed by some benchmark results, but rather because the qualitative experience was interesting, even fascinating! So for those itching to break out of the objective paradox, instead of thinking about how to ace a test (as tempting as that is), try to imagine what experience would be similarly fascinating to the general public without needing any benchmark result at all. That is the path of the interesting instead of the path of the objective, and indeed we often collapse to something like only 1% of effort given to the path of the interesting. Those 99% who are trapped wallowing in false security blanket of the objective paradox dwell there at their peril.
-
Kenneth Stanley reposted
Reads with Ravi
@readswithravi
This book is a mind-blowing read.
-
Kenneth Stanley reposted
hardmaru
@hardmaru
“AI creating AI”
2024 was a blast. @SakanaAILabs has been in operation for just over a year, and we have built not only a world class research team with some of the most creative and innovative minds, but also a world class applied engineering and business team in Tokyo. I feel as a founder, recruiting 🐟 is the most important and most critical part of my job. (sakana.ai/careers/)
I believe the path to develop much stronger AI systems is to automate the development of AI using AI. Sakana AI aims to develop AI systems that can create even more capable and efficient AI systems.
To that end, I’m so proud of my team this year, who have worked tirelessly to push the frontiers of our R&D. As a quick recap, here are some selected works from Sakana AI and our collaborators, related to “AI creating AI” which has been released publicly:
AI systems that can automate the creation of new foundation models with new capabilities, in Evolutionary Model Merge and CycleQD:
sakana.ai/evolutionary-model-merge/
sakana.ai/cycleqd/
Can we also use LLMs to come up with a much better algorithm to train LLMs themselves? We showed that this is possible. We term this self-referential improvement process LLM² (‘LLM-squared’) as an homage to previous fundamental work in meta-learning.
sakana.ai/llm-squared/
The AI Scientist: an AI system for automating scientific research and open-ended discovery, applied to AI research. We envisioned a system capable of tasks ranging from ideation, writing code, running experiments and summarizing results, to writing entire papers and conducting peer-review. We believe this work opens up a new era of AI-driven scientific research and accelerated discovery.
sakana.ai/ai-scientist/
AI to automate the creation of new primitive forms of Artificial Life, in our work on Automating the Search for Artificial Life with Foundation Models (ASAL).
sakana.ai/asal/
We have so much more planned for 2025, please stay tuned. 🔥🔥🔥
Thank you for all your support. -
Kenneth Stanley reposted
Akarsh Kumar
@akarshkumar0101
Very excited to share ASAL!
Artificial Life aims to recreate natural evolution, but is severely bottlenecked by hand-designed simulations.
We propose using CLIP to automatically discover the interesting ALife simulations!
Sakana AI
@SakanaAILabs
Introducing ASAL: Automating the Search for Artificial Life with Foundation Models
sakana.ai/asal/
Artificial Life (ALife) research holds key insights that can transform and accelerate progress in AI. By speeding up ALife discovery with AI, we accelerate our understanding of emergence, evolution, and intelligence–core principles that can inspire the next generation of AI systems!
We proudly collaborated with MIT, OpenAI, Swiss AI Lab IDSIA, and Ken Stanley on this exciting project.
Full Paper (Website): pub.sakana.ai/asal/
Full Paper (arxiv): asal.sakana.ai/paper/
Code: github.com/SakanaAI/asal/
In this work, we propose a new algorithm called Automated Search for Artificial Life (“ASAL”) to automate the discovery of artificial life using vision-language foundation models. Instead of tediously hand-designing every tiny rule of an Alife simulation, simply describe the space of simulations to search over, and ASAL will automatically discover the most interesting and open-ended artificial lifeforms!
Because of the generality of foundation models, ASAL can discover new lifeforms across a diverse range of seminal ALife simulations, including Boids, Particle Life, Game of Life, Lenia, and Neural Cellular Automata. ASAL even discovered novel cellular automata rules that are more open-ended and expressive than the original Conway’s Game of Life.
We believe this new paradigm may reignite ALife research by overcoming the bottleneck of manually designed simulations, thus advancing beyond the limits of human ingenuity.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
Very proud to have collaborated with @akarshkumar0101. @SakanaAILabs & others in a significant step forward for artificial life: guiding the search for artificial life with foundation models! Time for the AI revolution to pump some energy back into alife too!
Sakana AI
@SakanaAILabs
Introducing ASAL: Automating the Search for Artificial Life with Foundation Models
sakana.ai/asal/
Artificial Life (ALife) research holds key insights that can transform and accelerate progress in AI. By speeding up ALife discovery with AI, we accelerate our understanding of emergence, evolution, and intelligence–core principles that can inspire the next generation of AI systems!
We proudly collaborated with MIT, OpenAI, Swiss AI Lab IDSIA, and Ken Stanley on this exciting project.
Full Paper (Website): pub.sakana.ai/asal/
Full Paper (arxiv): asal.sakana.ai/paper/
Code: github.com/SakanaAI/asal/
In this work, we propose a new algorithm called Automated Search for Artificial Life (“ASAL”) to automate the discovery of artificial life using vision-language foundation models. Instead of tediously hand-designing every tiny rule of an Alife simulation, simply describe the space of simulations to search over, and ASAL will automatically discover the most interesting and open-ended artificial lifeforms!
Because of the generality of foundation models, ASAL can discover new lifeforms across a diverse range of seminal ALife simulations, including Boids, Particle Life, Game of Life, Lenia, and Neural Cellular Automata. ASAL even discovered novel cellular automata rules that are more open-ended and expressive than the original Conway’s Game of Life.
We believe this new paradigm may reignite ALife research by overcoming the bottleneck of manually designed simulations, thus advancing beyond the limits of human ingenuity.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Sakana AI
@SakanaAILabs
Introducing ASAL: Automating the Search for Artificial Life with Foundation Models
sakana.ai/asal/
Artificial Life (ALife) research holds key insights that can transform and accelerate progress in AI. By speeding up ALife discovery with AI, we accelerate our understanding of emergence, evolution, and intelligence–core principles that can inspire the next generation of AI systems!
We proudly collaborated with MIT, OpenAI, Swiss AI Lab IDSIA, and Ken Stanley on this exciting project.
Full Paper (Website): pub.sakana.ai/asal/
Full Paper (arxiv): asal.sakana.ai/paper/
Code: github.com/SakanaAI/asal/
In this work, we propose a new algorithm called Automated Search for Artificial Life (“ASAL”) to automate the discovery of artificial life using vision-language foundation models. Instead of tediously hand-designing every tiny rule of an Alife simulation, simply describe the space of simulations to search over, and ASAL will automatically discover the most interesting and open-ended artificial lifeforms!
Because of the generality of foundation models, ASAL can discover new lifeforms across a diverse range of seminal ALife simulations, including Boids, Particle Life, Game of Life, Lenia, and Neural Cellular Automata. ASAL even discovered novel cellular automata rules that are more open-ended and expressive than the original Conway’s Game of Life.
We believe this new paradigm may reignite ALife research by overcoming the bottleneck of manually designed simulations, thus advancing beyond the limits of human ingenuity.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
hardmaru
@hardmaru
Automating the Search for Artificial Life with Foundation Models asal.sakana.ai/paper/
We propose a new method called Automated Search for Artificial Life (ASAL) which uses foundation models to automate the discovery of the most interesting and open-ended artificial lifeforms.
Sakana AI
@SakanaAILabs
Introducing ASAL: Automating the Search for Artificial Life with Foundation Models
sakana.ai/asal/
Artificial Life (ALife) research holds key insights that can transform and accelerate progress in AI. By speeding up ALife discovery with AI, we accelerate our understanding of emergence, evolution, and intelligence–core principles that can inspire the next generation of AI systems!
We proudly collaborated with MIT, OpenAI, Swiss AI Lab IDSIA, and Ken Stanley on this exciting project.
Full Paper (Website): pub.sakana.ai/asal/
Full Paper (arxiv): asal.sakana.ai/paper/
Code: github.com/SakanaAI/asal/
In this work, we propose a new algorithm called Automated Search for Artificial Life (“ASAL”) to automate the discovery of artificial life using vision-language foundation models. Instead of tediously hand-designing every tiny rule of an Alife simulation, simply describe the space of simulations to search over, and ASAL will automatically discover the most interesting and open-ended artificial lifeforms!
Because of the generality of foundation models, ASAL can discover new lifeforms across a diverse range of seminal ALife simulations, including Boids, Particle Life, Game of Life, Lenia, and Neural Cellular Automata. ASAL even discovered novel cellular automata rules that are more open-ended and expressive than the original Conway’s Game of Life.
We believe this new paradigm may reignite ALife research by overcoming the bottleneck of manually designed simulations, thus advancing beyond the limits of human ingenuity.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
New AGI benchmark proposal: instead of giving the AI a hard test like AIME, have the AI come up with a corresponding hard test of its own (and answer key), and then give that test to human students of the right level. Was it a good test with novel questions?
Kenneth Stanley
@kenneth0stanley
Not only will this benchmark help us understand the AI’s human-level creative abilities, but it will eventually save real human effort - if it succeeds then much less need for the committees who write these tests!
-
Kenneth Stanley reposted
Machine Learning Street Talk
@MLStreetTalk
This is a clip of professor @jeffclune - amazing interview at #neurips2024 on the thesis of open-endedness research in AI. Over the moon that we finally got to interview Jeff! See below for some recent papers 👇
This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Machine Learning Street Talk
@MLStreetTalk
Being able to define "interestingness" is the grand challenge in open-endedness, and indeed, perhaps intelligence! -- with @jeffclune (this interview is gold 😃)
This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Liron Shapira
@liron
KEN STANLEY AI DOOM DEBATE
This week on Doom Debates, I'm interviewing and debating Prof. Kenneth Stanley (@kenneth0stanley)!
Ken led the Open-Endedness research team at @OpenAI from 2020-22. Before that he was a Professor of Computer Science at the University of Central Florida and was head of Core AI research at Uber.
He coauthored _Why Greatness Cannot Be Planned: The Myth of the Objective_, which argues that as soon as you create an objective, you ruin your ability to reach it. He's also known for creating the Neuroevolution of Augmenting Topologies (NEAT) algorithm.
I believe greatness *can* be planned and objectives can be successfully achieved by AGIs, so that'll be my position in the debate 😂
We'll also attempt to nail down why he isn't an AI doomer.
Feel free to post any suggested questions and topics below!
(Note: Roon episode is already recorded and will drop very soon. The Ken episode will be after that.) -
Kenneth Stanley
@kenneth0stanley
Buying a limestone mine with no objective (though interesting!) - that’s definitely in the spirit of Why Greatness Cannot Be Planned!
Andrew Ellis
@amellis_84
“Why did you guys buy an old limestone? What are you planning to do with 2,000 underground acres?”
People are baffled by my answer… “I don’t know”.
Confusion becomes intrigue when I rattle of a handful of fanciful things we’re testing and considering.
Re: @kenneth0stanley -
Kenneth Stanley
@kenneth0stanley
Evolution had a good run evolving human-level intelligence the first time around - don’t count it out now in round 2! (Congrats to @SakanaAILabs for provocative work!)
Sakana AI
@SakanaAILabs
Introducing An Evolved Universal Transformer Memory
sakana.ai/namm
Neural Attention Memory Models (NAMMs) are a new kind of neural memory system for Transformers that not only boost their performance and efficiency but are also transferable to other foundation models, without any additional training!
Full Paper: arxiv.org/abs/2410.13166
Code: github.com/SakanaAI/evo-memory
Memory is a crucial component of cognition, allowing humans to selectively store and extract important notions from our ceaseless exposure to information and noise. Our work learns an artificial memory to replicate these capabilities thanks to the power of evolution, leading to smarter, faster, and more adaptable foundation models. -
Kenneth Stanley reposted
Sakana AI
@SakanaAILabs
Introducing An Evolved Universal Transformer Memory
sakana.ai/namm
Neural Attention Memory Models (NAMMs) are a new kind of neural memory system for Transformers that not only boost their performance and efficiency but are also transferable to other foundation models, without any additional training!
Full Paper: arxiv.org/abs/2410.13166
Code: github.com/SakanaAI/evo-memory
Memory is a crucial component of cognition, allowing humans to selectively store and extract important notions from our ceaseless exposure to information and noise. Our work learns an artificial memory to replicate these capabilities thanks to the power of evolution, leading to smarter, faster, and more adaptable foundation models. -
Kenneth Stanley
@kenneth0stanley
One day long ago, someone bred a butterfly on an obscure picture-breeding website. More than 10 years later over half a million people see that same butterfly on X. Just like the title says, that butterfly’s adventure was totally unplanned! (Thanks @Ashwinreads for call out!)
Ashwin Sharma
@Ashwinreads
currently having my mind blown reading this book in a coffee shop. seriously wtf, wish i found this sooner.
-
Kenneth Stanley reposted
Ashwin Sharma
@Ashwinreads
currently having my mind blown reading this book in a coffee shop. seriously wtf, wish i found this sooner.
-
Kenneth Stanley reposted
Alex Havrilla
@Dahoas1
How important is the quality, diversity, and complexity (QDC) of synthetic data for LLM performance? What effect does QDC data composition have on self-improvement?
We just released a comprehensive survey discussing these questions (and many more) 🧵 -
Kenneth Stanley reposted
Jack Parker-Holder
@jparkerholder
Introducing 🧞Genie 2 🧞 - our most capable large-scale foundation world model, which can generate a diverse array of consistent worlds, playable for up to a minute. We believe Genie 2 could unlock the next wave of capabilities for embodied agents 🧠.
-
Kenneth Stanley
@kenneth0stanley
I had a great conversation on the @ThruConvPodcast , thank you @alex_levox for hosting!
Through Conversations Podcast
@ThruConvPodcast
In this episode, @kenneth0stanley explores the contrast between objective-driven living and open-ended exploration, emphasizing serendipity, creativity, and the value of risk in pursuing interests. He critiques the stifling effects of objective-based education and discusses AI's potential to enhance creativity, eliminate monotony, and reshape human ambition. Stanley also raises profound questions about AI's impact on creativity and our aspirations.
Full episode - www.youtube.com/watch?v=eEdPuTAQ1nU
Kenneth O. Stanley, previously CEO of Maven, is an AI researcher known for NEAT, co-author of "Why Greatness Cannot Be Planned," and has worked at Uber AI Labs and OpenAI.
Highlights
0:00:00 Introduction & Episode Preview
0:06:20 Meaning of “Never-Ending Algorithm” in Intelligence
0:11:00 Experiment “Pick a Breeder” from Why Greatness Cannot Be Planned
0:16:30 Serendipity: A Window for the World
0:23:56 Risk and Intuition in Pursuing Interests
0:31:17 Redefining Success in Modern Society
0:39:04 Tuning More Into Our Intuition
0:46:34 AI: Creativity vs. Job Displacement
1:00:17 AI and Freeing Time for Creativity -
Kenneth Stanley
@kenneth0stanley
How to know when you’re hitting a wall*: watch for when the benchmark rigor police start coming out in force. It happens in every epoch of AI research. Whenever the gains of a paradigm slow, the rigor police reemerge in the vain hope that the objective paradox can be thwarted.
*Important caveat: this is not a fatalistic take - it’s possible to find a way around walls, or even to break through them. But it’s still very useful to know where they are.
Kenneth Stanley
@kenneth0stanley
The objective paradox (as discussed in depth in our book Why Greatness Cannot Be Planned): the problem that the more you focus (or even obsess) on a highly ambitious objective, paradoxically, the less likely you become to achieve it.
-
Kenneth Stanley
@kenneth0stanley
While they disagree on some things, it’s interesting that both @ESYudkowsky and @stephen_wolfram infer broad conclusions from the unquestioned premise that evolution is primarily an optimization algorithm. For @ESYudkowsky, a dire succession of probable negative events begins with the assumption that a higher intelligence is inevitably relentlessly perfecting an optimization process towards an objective in what he believes is the spirit and implication of an evolutionary process. For @stephen_wolfram, the arbitrary happenstance of relentless optimization towards an objective yields bizarre intermediate constructs that defy intuition. But the premise is wrong in both cases because the force of optimization pales in comparison to divergent open-endedness in natural evolution (and creative processes general). This idea that evolution is an optimizer akin to objective-driven gradient descent, which early simple genetic algorithms inadvertently perpetuated, is still leading even the best thinkers astray without any challenge.
Consider: a super-intelligence that rivals human creativity cannot primarily work through optimization because creativity requires open-endedness, which is divergent and not convergent in its “thinking.” With the assumption of the primacy of optimization tossed out, numerous downstream implications about how such a system would behave or how its products are assembled need to be revisited.
Note that I’m not implying that everyone’s conclusions are necessarily all wrong - you can still come to a correct conclusion despite a flawed premise, and caution about AI’s future impact is warranted - but it would still clearly be better if the argument started with a correct premise. Unchained open-endedness also carries its own risks, which deserve serious analysis of their own, but it drew not even a moment of attention in this 4-hour epic.
In any case, congrats to @MLStreetTalk on bringing together such provocative and brilliant thinkers into conversations with each other - more such surprise pairings would be great!
Machine Learning Street Talk
@MLStreetTalk
Conversation with @ESYudkowsky and @stephen_wolfram live on MLST now.
-
Kenneth Stanley
@kenneth0stanley
You know you have a good friend when they nominate you for a Nobel Prize! @jeffclune could have a Turing Award in his future himself! Deep learning is having its well-deserved moment but open-endedness has plenty of runway ahead of it, and it may be just the friend DL needs.
Jeff Clune
@jeffclune
Love this idea. Picbreeder for OMNI-EPIC sounds great to me! Also love how often I am reminded of what a pioneering genius @kenneth0stanley is. When does he get his Nobel prize?
-
Kenneth Stanley
@kenneth0stanley
With much fretting on whether “scaling laws” are plateauing, it’s important to remember that the moment you realize the limitations of a thing can be the essential inflection to seeing once hidden paths of gold. And I don’t mean “throwing out” what we’ve gained - when I invented NEAT I did not “throw out” evolutionary algorithms - rather I saw how their scope could widen. If the fruits of scale are indeed blossoming slower then a monumental opportunity is looming. What we’ve learned about data and scale in neural networks remains a profound and powerful stepping stone that cannot be unlearned, but we will begin to see (just barely now) the hints of even deeper and more disruptive truths beneath that.
-
Kenneth Stanley
@kenneth0stanley
Two things that both evolution and neural networks share in common is they both thrive on scale, and they were both once dismissed as obsolete to AI. The deep evolution moment is awaiting.
François Chollet
@fchollet
Similarly, there are several research directions today seen as long-abandoned failures, that are only waiting for the right amount of attention and the right level of compute scale to shine. Genetic algorithms are one of them.
-
Kenneth Stanley reposted
Cristóbal Valenzuela
@c_valenzuelab
There is a dangerous habit of drawing straight lines. The predominant paradigm of research and benchmarks assumes a clean transition from one modality (e.g., text) to another (e.g., image). From a purely research perspective, this makes sense - you can quantify improvements in convenient and neat ways. It's tidy. It's logical. But for creative purposes and actual media applications, it's completely wrong.
I think this paradigm leads to building systems based on a fundamental misunderstanding of how creation actually works. It treats creative work as if it were an assembly line: input goes in one end, output comes out the other. Text becomes image becomes video, as if creativity were just a matter of format conversion.
But real creation isn't linear. It's not even planar. It's a complex web of decisions, revisions, and possibilities. Each choice opens up new paths, creates new connections, generates new possibilities. The creative process looks more like a neural network than a flowchart - a tangled mess of interconnected decisions and possibilities.
Format-conversion-driven development is a misunderstanding of how to best approach AI research aimed at creative outputs. It leads to building pipelines when it should be building playgrounds. It optimizes for direct paths when it should be optimizing for exploration. It simplifies when it should be embracing complexity.
The most powerful creative tools aren't the ones that move you efficiently from point A to point B. They're the ones that help you discover point C, D, and E along the way. They're the ones that treat creation as exploration rather than execution.
Think about how most artists actually work. They don't follow a straight path from concept to completion. They sketch, they erase, they revise, they start over. They make choices that lead to dead ends, which inspire new directions. They discover their destination by wandering. The predominant paradigm of AI tools often ignores this reality, optimizing for destination when it should be optimizing for discovery.
Embrace complexity. Instead of trying to simplify the creative process into linear paths, I encourage creating tools that support the messy, non-linear nature of real creation. Tools that encourage exploration, that treat each choice point as an opportunity for discovery rather than just a step toward a predetermined goal. The focus shouldn't be on reducing choices, but on multiplying them. The aim should be to build systems that understand and support the complex, interconnected nature of real creative work - systems that don't just execute commands but help explore possibilities. -
Kenneth Stanley
@kenneth0stanley
It was a lot of fun speaking about the objective paradox on the Theory of Anything podcast - Bruce and Peter asked deep and pointed questions that go well beyond just introducing the main ideas. Thanks to @bnielson01 for having me!
Bruce Nielson
@bnielson01
We interview AI researcher @kenneth0stanley. He makes the case that pursing specific objectives can be counterproductive. Instead, whether in AI, business, science, education, or art, we should pursue what is interesting.
open.spotify.com/episode/5ZxMeMySrTnKsCvSTytIbA -
Kenneth Stanley
@kenneth0stanley
Great summary thread of Why Greatness Cannot Be Planned from @paraschopra !
Paras Chopra
@paraschopra
How to achieve success without trying to shoot for it!
My notes from the #book "Why Greatness Cannot be Planned" by @kenneth0stanley and @joelbot3000
🧵 -
Kenneth Stanley reposted
Paras Chopra
@paraschopra
How to achieve success without trying to shoot for it!
My notes from the #book "Why Greatness Cannot be Planned" by @kenneth0stanley and @joelbot3000
🧵 -
Kenneth Stanley reposted
Tim Rocktäschel
@_rockt
I had a fantastic time talking with @samcharrington (@twimlai) about @orionbooks' "Artificial Intelligence: 10 Things You Should Know" (geni.us/ArtificialIntelligence) and many exciting 2024 research papers (some of them from my teams) in the Open-Endedness community by outstanding researchers like @merrierm @jennyzhangzt @jeffclune @chrisantha_f @2ne1 @edwardfhughes @MichaelD1729 @ber24 @akbirkhan @_chris_lu_ @cong_ml @RobertTLange @j_foerst @hardmaru ...
Levels of AGI: Morris et al. Levels of AGI: Operationalizing Progress on the Path to AGI. ICML 2024. doi.org/10.48550/arXiv.2311.02462
OMNI: Zhang et al. OMNI: Open-endedness via Models of human Notions of Interestingness. ICML 2024. arxiv.org/abs/2306.01711
Promptbreeder: Fernando et al. Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution. ICML 2024. arxiv.org/abs/2309.16797
Epistemology (@DavidDeutschOxf): Deutsch, D. (2012). The Beginning of Infinity. Penguin Books.
Open-Endedness: Hughes et al. Open-Endedness is Essential for Artificial Superhuman Intelligence. ICML 2024. arxiv.org/abs/2406.04268
Rainbow Teaming: Samvelyan et al. Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts. NeurIPS 2024. doi.org/10.48550/arXiv.2402.16822
FunSearch: Romera-Paredes et al. Mathematical discoveries from program search with large language models. Nature, 625(7995), 468–475. doi.org/10.1038/s41586-023-06924-6
AI Debate: Khan et al. Debating with More Persuasive LLMs Leads to More Truthful Answers. ICML 2024. arxiv.org/abs/2402.06782
AI Scientist: Lu et al. The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery. arXiv 2024. doi.org/10.48550/arXiv.2408.06292
The TWIML AI Podcast
@twimlai
Today, we're joined by @_rockt, senior staff research scientist at @GoogleDeepMind, professor of AI at @ai_ucl, and author of the recently published book, “Artificial Intelligence: 10 Things You Should Know.” We dig into the attainability of artificial superintelligence and the path to achieving generalized superhuman capabilities across multiple domains. We discuss the importance of open-endedness in developing autonomous and self-improving systems, and the role of evolutionary approaches and algorithms. Additionally, we cover Tim’s recent research projects such as “Promptbreeder,” “Debating with More Persuasive LLMs Leads to More Truthful Answers,” and more.
🎧 / 🎥 Listen or watch the full episode on our page: twimlai.com/go/706.
📖 CHAPTERS
===============================
00:00 - Introduction
2:38 - Book: Artificial Intelligence: 10 Things You Should Know
4:42 - Artificial superhuman intelligence
12:23 - Achieving ASI
16:15 - Path to ASI
20:44 - Reward functions
23:10 - Prompt strategies
30:58 - Evolutionary approaches
36:29 - Taxonomies of LLM evaluation function
39:52 - Automating Debate paper
47:29 - Inference scaling
50:13 - Future directionsThis media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Through Conversations Podcast
@ThruConvPodcast
Why planning your life will ruin your success (full solo episode on @X)
Highlights
00:00 Introduction and Episode Preview
01:53 AI and Life Lessons
02:02 Analyzing "Why Greatness Cannot Be Planned"
06:09 Success Through Less Goal Focus
10:54 The Dangers of Goal Setting
13:00 Jim Carrey’s Advice
16:05 How to Pre-order "The Time is Now"This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Tim Rocktäschel
@_rockt
Had a great conversation with @ecsquendor back at ICML, covering multiple recent papers
Open-Endedness: arxiv.org/abs/2406.04268
Genie: arxiv.org/abs/2402.15391
Promptbreeder: arxiv.org/abs/2309.16797
Rainbow Teaming: arxiv.org/abs/2402.16822
AI Debate: arxiv.org/abs/2402.06782
Machine Learning Street Talk
@MLStreetTalk
🧠 Is the future of AI open-ended? Latest show with @_rockt (who achieved two ICML best paper awards! 📷youtu.be/6DrCq8Ry2cw
-
Kenneth Stanley
@kenneth0stanley
Recent results like Apple’s show that LLMs (even o1) flub on reasoning with simple changes to problems that shouldn’t matter. A consensus is building that it shows they are “just pattern matching.” But that metaphor is misleading: good reasoning itself can also be framed as “just pattern matching” at each step. The issue is not that we are merely seeing pattern matching, but that we are seeing *bad* pattern matching, at the wrong level of abstraction. If you think about it, that is a more serious pathology because it doesn’t separate when it works vs. when it doesn’t work into conveniently distinct buckets of computational tasks, In a sense, calling it “just pattern matching” implies an easier fix than there really is, as if all it will take is a better o1.
-
Kenneth Stanley
@kenneth0stanley
Is going into AI a good plan for becoming a Nobel laureate in chemistry? Why Greatness Cannot Be Planned! (Congrats to @demishassabis )
The Nobel Prize
@NobelPrize
BREAKING NEWS
The Royal Swedish Academy of Sciences has decided to award the 2024 #nobelprize in Chemistry with one half to David Baker “for computational protein design” and the other half jointly to Demis Hassabis and John M. Jumper “for protein structure prediction.” -
Kenneth Stanley
@kenneth0stanley
A common source of confusion about progress in AI is the tendency to conflate what we’ve built with what we’ve learned. The engine of progress is always what we’ve learned. And while we have indeed learned some deep lessons, what we’ve built is only a single narrow window.
-
Kenneth Stanley
@kenneth0stanley
Though useful and powerful, an LLM today trained on virtually all the text on the internet is still far from human-level. Do you think there exists *any* architecture and algorithm that when trained on this same corpus would surpass humans? Or is the data simply inadequate?
-
Kenneth Stanley reposted
Machine Learning Street Talk
@MLStreetTalk
🧠 Is the future of AI open-ended? Latest show with @_rockt (who achieved two ICML best paper awards! 📷youtu.be/6DrCq8Ry2cw
-
Kenneth Stanley reposted
Maxence Faldor
@maxencefaldor
Interested in emergence, self-organization or morphogenesis?
Introducing CAX: Cellular Automata Accelerated in JAX, a high-performance and flexible library designed to accelerate cellular automata research. 🦎
📄: arxiv.org/abs/2410.02651
🌟: github.com/maxencefaldor/cax
🧵1/12 -
Kenneth Stanley
@kenneth0stanley
Prediction: while backprop will likely play a role in any future AGI, there will also be at least one completely novel weight-updating learning algorithm that is yet to be invented today.
-
Kenneth Stanley reposted
Julian Togelius
@togelius
Today is the official release day for my little book on Artificial General Intelligence, published by MIT Press. It's available on the shelf of well-stocked booksellers, and I wrote it to be accessible to as large audience as possible; it's not really a technical book, even though it tackles some technical topics. I started working on this book about two years ago, and much has happened in the AI space since then. Still, I think it holds up well.
One of the main points is that artificial general intelligence is a confused and confusing idea, largely because we don't know what either intelligence or generality means. We keep making impressive progress in AI technology - and I try to explain some key AI methods, such as LLMs, in simple terms - but the various AI methods have different upsides and downsides, and we are far from having a single system that can do everything we think of as needing "intelligence". Clearly, the future of AI has room for many perspectives and different technical approaches. The book also discusses what more progress in AI could mean for society, and draws on science fiction to paint contrasting visions of what AGI might mean.
This has been a passion project of mine that I ended up using much of my sabbatical on. I'm an optimist, and I argue for open access to knowledge and technology, and against undue regulations. If I can achieve anything with this book, I hope that it will be to explain some of the wonderful possibilities of this technology to people, as it is natural to be afraid of things you don't understand. -
Kenneth Stanley reposted
Max | Emergent Garden
@max_romana
⛏️🧠Mindcraft update! Agents can now be assigned goals that they endlessly work towards. Includes a discussion of Open Endedness and endless games
youtu.be/IeXadWbvDiE -
Kenneth Stanley
@kenneth0stanley
The definition police will disagree, but some words are useful specifically because they are ill-defined. Sometimes it helps to have a succinct way to refer to a vague notion. Arguing about precise meaning is often less productive than actually doing something.
-
Kenneth Stanley
@kenneth0stanley
A very cool idea from @max_romana - Picbreeder plus LLMs for breeding interactive (or not) websites with visuals. You can explore this complicated space effortlessly with no need to code anything. Worth trying out!
Max | Emergent Garden
@max_romana
evolvecode.io
generate -> select -> modify -> repeat
I made this a few months ago, it uses LLMs to generate many websites that you can then select/modify endlessly. picbreeder but for code! (@kenneth0stanley) -
Kenneth Stanley
@kenneth0stanley
A major conundrum faced by AI investors is that while LLMs are a breakthrough technology, they are almost certainly not the best architecture possible (the first breakthrough in any tech almost never is). So as investment stakes go up, with $100B runs on the table, it becomes increasingly unclear whether the best investment is to double down all the way to the unfathomably expensive local optimum, or to back the more speculative but likely longer-run essential search for even better architectures. Given the increasingly high stakes and unpredictability, some investors will ultimately prevail by putting money on the search for the disrupter architecture rather than by picking the winner of the current race.
-
Kenneth Stanley
@kenneth0stanley
Congrats @hardmaru and @SakanaAILabs ! Sakana is doing something unique within the world of AI!
hardmaru
@hardmaru
Excited to announce our Series A!
We raised more than $100M to grow Sakana AI into a World Class AI Lab in Japan. We’re going to really push the frontiers of what’s possible with AI.
As a founder mode startup, we operate much faster than most frontier AI labs at a global level. -
Kenneth Stanley reposted
Moein Shariatnia
@MoeinShariatnia
On the absurdity of peer-review:
"After all, attracting a unanimous vote in science could be a sign of nothing more than echoing the status quo.
If you’re doing whatever is hot and parrot the right buzzwords, you might be able to attract wide support"
Can't agree more -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Can AI agents design better AI agents?
We describe a newly forming research area "Automated Design of Agentic Systems" (ADAS) that aims to automatically design powerful agents & a new method where a meta agent invents new agents by programming them in code. Led by @shengranhu 🧵 -
Kenneth Stanley reposted
Big Think
@bigthink
Do things that are simply interesting in their own right. (@kenneth0stanley)
-
Kenneth Stanley
@kenneth0stanley
Inspiring to imagine open-ended algorithms applied to a genuine real-world domain like scientific research. This is why the field open-endedness will be increasingly indispensable within AI. Congrats to @hardmaru @jeffclune et al!
Sakana AI
@SakanaAILabs
Introducing The AI Scientist: The world’s first AI system for automating scientific research and open-ended discovery!
sakana.ai/ai-scientist/
From ideation, writing code, running experiments and summarizing results, to writing entire papers and conducting peer-review, The AI Scientist opens a new era of AI-driven scientific research and accelerated discovery.
Here are 4 example Machine Learning research papers generated by The AI Scientist.
We published our report, The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery, and open-sourced our project!
Paper: arxiv.org/abs/2408.06292
GitHub: github.com/SakanaAI/AI-Scientist
Our system leverages LLMs to propose and implement new research directions. Here, we first apply The AI Scientist to conduct Machine Learning research. Crucially, our system is capable of executing the entire ML research lifecycle: from inventing research ideas and experiments, writing code, to executing experiments on GPUs and gathering results. It can also write an entire scientific paper, explaining, visualizing and contextualizing the results.
Furthermore, while an LLM author writes entire research papers, another LLM reviewer critiques resulting manuscripts to provide feedback to improve the work, and also to select the most promising ideas to further develop in the next iteration cycle, leading to continual, open-ended discoveries, thus emulating the human scientific community. As a proof of concept, our system produced papers with novel contributions in ML research domains such language modeling, Diffusion and Grokking.
We (@_chris_lu_, @RobertTLange, @hardmaru) proudly collaborated with the @UniOfOxford (@j_foerst, @FLAIR_Ox) and @UBC (@cong_ml, @jeffclune) on this exciting project. -
Kenneth Stanley reposted
Sakana AI
@SakanaAILabs
Introducing The AI Scientist: The world’s first AI system for automating scientific research and open-ended discovery!
sakana.ai/ai-scientist/
From ideation, writing code, running experiments and summarizing results, to writing entire papers and conducting peer-review, The AI Scientist opens a new era of AI-driven scientific research and accelerated discovery.
Here are 4 example Machine Learning research papers generated by The AI Scientist.
We published our report, The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery, and open-sourced our project!
Paper: arxiv.org/abs/2408.06292
GitHub: github.com/SakanaAI/AI-Scientist
Our system leverages LLMs to propose and implement new research directions. Here, we first apply The AI Scientist to conduct Machine Learning research. Crucially, our system is capable of executing the entire ML research lifecycle: from inventing research ideas and experiments, writing code, to executing experiments on GPUs and gathering results. It can also write an entire scientific paper, explaining, visualizing and contextualizing the results.
Furthermore, while an LLM author writes entire research papers, another LLM reviewer critiques resulting manuscripts to provide feedback to improve the work, and also to select the most promising ideas to further develop in the next iteration cycle, leading to continual, open-ended discoveries, thus emulating the human scientific community. As a proof of concept, our system produced papers with novel contributions in ML research domains such language modeling, Diffusion and Grokking.
We (@_chris_lu_, @RobertTLange, @hardmaru) proudly collaborated with the @UniOfOxford (@j_foerst, @FLAIR_Ox) and @UBC (@cong_ml, @jeffclune) on this exciting project. -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Introducing The AI Scientist! 🧪🔬🔭It creates research ideas & experiments, any necessary code, runs experiments, plots & analyzes data, writes an ENTIRE science manuscript, & performs peer review! Then builds on "published" discoveries.Fully automated. A new era in science?🧵👇
Sakana AI
@SakanaAILabs
Introducing The AI Scientist: The world’s first AI system for automating scientific research and open-ended discovery!
sakana.ai/ai-scientist/
From ideation, writing code, running experiments and summarizing results, to writing entire papers and conducting peer-review, The AI Scientist opens a new era of AI-driven scientific research and accelerated discovery.
Here are 4 example Machine Learning research papers generated by The AI Scientist.
We published our report, The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery, and open-sourced our project!
Paper: arxiv.org/abs/2408.06292
GitHub: github.com/SakanaAI/AI-Scientist
Our system leverages LLMs to propose and implement new research directions. Here, we first apply The AI Scientist to conduct Machine Learning research. Crucially, our system is capable of executing the entire ML research lifecycle: from inventing research ideas and experiments, writing code, to executing experiments on GPUs and gathering results. It can also write an entire scientific paper, explaining, visualizing and contextualizing the results.
Furthermore, while an LLM author writes entire research papers, another LLM reviewer critiques resulting manuscripts to provide feedback to improve the work, and also to select the most promising ideas to further develop in the next iteration cycle, leading to continual, open-ended discoveries, thus emulating the human scientific community. As a proof of concept, our system produced papers with novel contributions in ML research domains such language modeling, Diffusion and Grokking.
We (@_chris_lu_, @RobertTLange, @hardmaru) proudly collaborated with the @UniOfOxford (@j_foerst, @FLAIR_Ox) and @UBC (@cong_ml, @jeffclune) on this exciting project. -
Kenneth Stanley
@kenneth0stanley
It's interesting that HyperNEAT, with its geometry-based indirect NN encoding and variable resolution, looks particularly well-suited to ARC-like problems. I'd guess it's not really the kind of solution @fchollet has in mind, but there's a kernel of insight there.
-
Kenneth Stanley reposted
Jean-Baptiste Mouret
@jb_mouret
Very happy to see our Map-elites algorithm used in one of the most important ML project of our times!
Mikayel Samvelyan
@_samvelyan
Excited to share that our 🌈 Rainbow Teaming method has been used to evaluate and enhance the adversarial robustness of Llama 3.1 models!
Originally an exploratory project co-led with @_andreilupu and @sharathraparthy, it has contributed to the biggest release of the year! -
Kenneth Stanley
@kenneth0stanley
Some major personal news: I’m moving on from Maven and exploring new opportunities. Even so, Maven is not going anywhere and will continue to be a beacon of serendipity in a world of popularity-driven media. So what happened? There is no doubt that the internet needs something like Maven, something to take us away from the endless convergent popularity contest and closer to serendipity. And indeed, people respond enthusiastically to that idea. But we could not achieve the kind of growth curve that investors want to see to justify increasing investment, and there is likely still a missing ingredient for that kind of growth.
So, to extend Maven's runway and give it the best shot of finding this right set of ingredients for real growth, my cofounders and I have decided to pare back the team, leaving our CTO, @jecretan, to drive product innovation and next steps. My cofounders (Jimmy and Blas) and I made this decision together and remain supportive of Jimmy and the mission, and (importantly!) remain friends with each other. Jimmy will henceforth drive product and innovation at the company.
I knew I took a significant risk to try to build a new kind of social network based on what I’d learned in my explorations of open-endedness. I still believe it was the right move to make for me, and that it’s the right move for social media. But more is needed to get the growth we need to truly challenge the status quo. I hope we still can get there.
In the meantime, I will return to AI, open-endedness, and new paradigms not yet imagined. Feel free to get in touch if you’re interested in my next direction!
Kenneth Stanley
@kenneth0stanley
Oops I tagged Jimmy wrongly - he's happy to hear from people about Maven going forward. You can find him at @jsecretan
Kenneth Stanley
@kenneth0stanley
Just made DMs open - several people noted they were trying to get in touch.
-
Kenneth Stanley reposted
Tim Taylor
@drtimt
It was fantastic to have the opportunity of a long discussion yesterday with the one and only @kenneth0stanley about open endedness and open-ended evolution. Many thanks to @barbalet for hosting on his Last Monkey Standing channel! youtu.be/MIi5JN3Zzb8
-
Kenneth Stanley
@kenneth0stanley
A very different conversation than the usual: I joined @barbalet for his Last Monkey Standing podcast, along with leading open-ended evolution researcher @drtimt, to discuss the connections and contrasts between AI & Alife in the field of open-endedness: www.youtube.com/watch?v=MIi5JN3Zzb8
-
Kenneth Stanley reposted
Julian Togelius
@togelius
Six years ago, @yannakakis and I published a textbook on AI and Games. It has been used in universities across the world and become a standard reference for the field. Much has happened since, and it is about time for a second edition. Good news: we just put a draft online!
-
Kenneth Stanley
@kenneth0stanley
The main reason that LLMs bring us closer to AGI is not that they are closer to AGI than what came before them (though they very well may be), but rather that they more clearly expose the holes that remain to be filled to achieve AGI, thereby bringing a novel clarity to the question of what kind of AI research is interesting. Those who seize this opportunity will be the ones to celebrate and obsess on the holes.
-
Kenneth Stanley reposted
Joel Lehman
@joelbot3000
"We Need Positive Visions for AI Grounded in Wellbeing"
Excited to share article written w/ great @manda_ngo in @gradientpub
Looking beyond techno-pessimism and optimism, we clarify "beneficial AI" for individuals + society & lay out concrete research directions.
Details in 🧵
The Gradient
@gradientpub
We hear a lot about “beneficial AI” or AI “aligned with human values” — but what does this mean and how do we get there? @joelbot3000 and @manda_ngo write: thegradient.pub/we-need-positive-visions-for-ai-grounded-…
-
Kenneth Stanley
@kenneth0stanley
A lot work in AI Safety today rests on the hidden assumption that the current prevailing AI paradigm is the final paradigm. But something new could easily subsume what we have now, either by building on it or replacing it, making all kinds of assumptions moot. This fact is no one's fault, and does not imply that today's work is without value. But it presents a real challenge for thinking forward.
-
Kenneth Stanley
@kenneth0stanley
AI progress will eventually hit a wall if it fails to confront the challenge of open-endedness. This work by @_rockt , @edwardfhughes et al. that boldly states that it is "essential" while bringing clarity and focus is an exciting contribution to this area.
Machine Learning Street Talk
@MLStreetTalk
This is @kenneth0stanley and @_rockt on the importance of open-endedness in AI research. @edwardfhughes, TimR and colleagues at @GoogleDeepMind have just written "Open-Endedness is Essential for Artificial Superhuman Intelligence"
This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Machine Learning Street Talk
@MLStreetTalk
This is @kenneth0stanley and @_rockt on the importance of open-endedness in AI research. @edwardfhughes, TimR and colleagues at @GoogleDeepMind have just written "Open-Endedness is Essential for Artificial Superhuman Intelligence"
This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
OMNI-EPIC & Intelligent Go-Explore in Science News! "Both works are significant advancements towards creating open-ended learning systems,” -Tim Rocktäschel.Lead by @jennyzhangzt @maxencefaldor & @conglu
www.sciencenews.org/article/ai-train-creative-human-intui… Quotes @j_foerst & @togelius too.Thx @SilverJacket! -
Kenneth Stanley reposted
Bryan Lim
@bryanlimwt
💡⚙️ Ideas and inventions are rarely generated and created in isolation 🏝️
We explore using few/many-shot prompting of language models with quality-diverse examples provided from QD algorithms for solution generation for QD problems!
Work done with @MFlageat @CULLYAntoine -
Kenneth Stanley reposted
andrew arruda
@andrewarruda
next read, “why greatness cannot be planned” by kenneth o. stanley and joel lehman
-
Kenneth Stanley
@kenneth0stanley
I'm curious, if you think "LLMs can't reason," what do you precisely mean by that and why do you think that is? In particular I'm curious about insights beyond "they are just doing statistics" or "they just make predictions." What beyond that do you think supports the claim? Note that I'm not trying to imply this statement is wrong or right, just curious what it reveals when people dig into the issue more deeply.
Kenneth Stanley
@kenneth0stanley
A few people have tried to infer from my OP my own position, so I wanted to give a little clarification on that: I don't really have a true/false position on whether LLMs can reason because I don't think the question is sufficiently clear to have any definitive answer. However, just because it doesn't have a clear answer doesn't mean it's uninteresting: I think it is actually a very interesting question, not for whatever true/false answer it elicits, but for the underlying assumptions it forces us to confront as we grapple with our intuitions. The intuition (of some people) that LLM's can't reason, while not necessarily clearly defined, is still there for a reason, and the reason it's felt so strongly for some people points to some very interesting (and actually diverse) assumptions that can be thought-provoking and lead to very useful discussion. I'm actually not very interested in arguing about definitions myself (e.g. "what is reasoning") - I don't mind fuzziness. But what I do find interesting is grappling with why intuitions are so strong in some cases even on issues that are in the end fuzzy. I believe there is insight to be gained in the tree of assumptions underneath that fuzziness.
-
Kenneth Stanley reposted
Laura Graesser
@lgraesser3
Huge congratulations to @kenneth0stanley @Evo_Daveo and @NeuralNets4Life for the well deserved recognition for HyperNEAT. 🥳👏🥳👏. Ken's right, David is an amazing researcher and colleague.
Kenneth Stanley
@kenneth0stanley
Alas I sadly tagged my coauthor David D'Ambrosio wrongly in the below post. David deserves recognition! Follow him @Evo_Daveo - he is one of the inventors of HyperNEAT and one of the best researchers I've ever had the pleasure of working with (and my very first Ph.D. student).
Kenneth Stanley
@kenneth0stanley
What an honor to receive this publication of the decade award from ALIFE and ISAL for my work on HyperNEAT with amazing coauthors @ddambrosio72 and @NeuralNets4Life ! Thank you to ISAL for this reminder that it's worth revisiting CPPNs, indirect encoding of neural networks, and HyperNEAT in this era of large models - these ideas take on new significance and new potential in today's context for their ability to encode massive structures in an entirely different way than today's prevailing paradigm. The next generation of models still stand to gain from the consequent creativity they can bring to the table. The original paper is available at stars.library.ucf.edu/cgi/viewcontent.cgi?article=3177&co…
ALIFE Conference 2027
@ALifeConf
The decade paper award goes to @kenneth0stanley et al. for Hypercube-Based Indirect Encoding for Evolving Large-Scale
Neural Networks aka HyperNEAT #alife2024 -
Kenneth Stanley
@kenneth0stanley
Alas I sadly tagged my coauthor David D'Ambrosio wrongly in the below post. David deserves recognition! Follow him @Evo_Daveo - he is one of the inventors of HyperNEAT and one of the best researchers I've ever had the pleasure of working with (and my very first Ph.D. student).
Kenneth Stanley
@kenneth0stanley
What an honor to receive this publication of the decade award from ALIFE and ISAL for my work on HyperNEAT with amazing coauthors @ddambrosio72 and @NeuralNets4Life ! Thank you to ISAL for this reminder that it's worth revisiting CPPNs, indirect encoding of neural networks, and HyperNEAT in this era of large models - these ideas take on new significance and new potential in today's context for their ability to encode massive structures in an entirely different way than today's prevailing paradigm. The next generation of models still stand to gain from the consequent creativity they can bring to the table. The original paper is available at stars.library.ucf.edu/cgi/viewcontent.cgi?article=3177&co…
ALIFE Conference 2027
@ALifeConf
The decade paper award goes to @kenneth0stanley et al. for Hypercube-Based Indirect Encoding for Evolving Large-Scale
Neural Networks aka HyperNEAT #alife2024 -
Kenneth Stanley
@kenneth0stanley
What an honor to receive this publication of the decade award from ALIFE and ISAL for my work on HyperNEAT with amazing coauthors @ddambrosio72 and @NeuralNets4Life ! Thank you to ISAL for this reminder that it's worth revisiting CPPNs, indirect encoding of neural networks, and HyperNEAT in this era of large models - these ideas take on new significance and new potential in today's context for their ability to encode massive structures in an entirely different way than today's prevailing paradigm. The next generation of models still stand to gain from the consequent creativity they can bring to the table. The original paper is available at stars.library.ucf.edu/cgi/viewcontent.cgi?article=3177&co…
ALIFE Conference 2027
@ALifeConf
The decade paper award goes to @kenneth0stanley et al. for Hypercube-Based Indirect Encoding for Evolving Large-Scale
Neural Networks aka HyperNEAT #alife2024 -
Kenneth Stanley
@kenneth0stanley
This is a thought-provoking statement against the boringness of perfect symmetry. But I think nature’s favorite trick is not so much asymmetry as “imperfect symmetry,” or what I’d call “repetition with variation,” like the fingers of the hand or the heart on one side of a mostly symmetry body. It’s a still a form of symmetry, but a more nuanced and (usefully) imperfect form of it. That’s what I was trying to capture with CPPNs (compositional pattern-producing. networks).
helena sarin
@NeuralBricolage
i find symmetry uninspiring, mechanical and boring so totally agree with the sentiment - 'Nature's perfection lies in its own asymmetry"
my work is the generative nature - starts as analog art processed through bespoke gen models chains and ends as the analog -
Kenneth Stanley
@kenneth0stanley
When you realize your book can still go viral 9 years after it was published. (Thanks @bmix012 !)
-
Kenneth Stanley reposted
Ruli Manurung
@rulimanurung
Agreed! This is another example of the principles expounded in "Why Greatness Cannot Be Planned" by @joelbot3000 and @kenneth0stanley! It goes by many names: serendipity, opportunistic planning, exploration, divergence. It's so central to our ability to flourish as a species!
-
Kenneth Stanley reposted
Sam Earle
@Smearle_RH
We introduce 𝘢𝘶𝘵𝘰𝘷𝘦𝘳𝘴𝘦 🌿🧫👽, a fast, evolvable description language for reinforcement learning environments, implemented in jax. First of many papers on this subject w/ @togelius.
Our goal is to provide a general framework for open-ended learning, in which we can implement canonical gridworld-type games, discover new (often alien) ones, and train robust player agents, that can observe a set of environment dynamics and adapt their behavior accordingly.
Notably, autoverse allows for the evolution of map layouts, rules and reward dynamics; other than the notion of an embodied "player" with the ability to edit tiles on the map, no semantics are hardcoded into the environment.
In sum, we set out to create a substrate for open-ended learning that is:
- capable of expressing a virtually limitless number of novel environments (via evolution of *all* aspects of the environment)
- fast, allowing for research at a small/academic scale (thanks to GPU implementation of environment dynamics)
- scalable, allowing for potentially unbounded environment complexity (by increasing the size of the map, number of rules, etc.)
Below is an example of an environment evolved via open-ended RL, in which environments are continually evolved to provide learnable challenges for player agents. Here, the agent is riding the waves generated by the environment, keeping them alive as long as it can to maximize reward. (The RL player is depicted in blue, game rules and their application across the board are depicted on the right.) -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Evolving Self-Assembling Neural Networks: From Spontaneous Activity to Experience-Dependent Learning 🧠🧬
Building on our previous works on Neural Developmental Programs (NDPs), we propose a class of self-organizing neural networks capable of synaptic and structural plasticity!
PDF: arxiv.org/abs/2406.09787 -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
We are happy to present "Meta-Learning an Evolvable Developmental Encoding"! 🧬
Generative models can work as learnable representations for blackbox optimization but they are not designed to be easily searchable.
We present a system that can meta-learn such representation by directly optimizing for a representation’s ability to generate quality diversity! -
Kenneth Stanley
@kenneth0stanley
A couple people tagged me on this (thanks!) - yet another story of Why Greatness Cannot Be Planned.
Prakash
@8teAPi
I’m sorry but the fact that AGI springs from a gaming chip company means that most state planning (thou shalt work on serious things!) and real growth fetishism (finance is sucking up our best brains) is wrong.
-
Kenneth Stanley
@kenneth0stanley
The spirit of Picbreeder in video in 2024.
Cristóbal Valenzuela
@c_valenzuelab
I've been watching too many people immerse themselves for hours using Gen-3, and there's this pattern that keeps popping up. It's like this:
You start with some vague idea in your head. But as you play around, you end up in totally different places. It's weird - the twists and turns become more interesting than what you first thought of. It's not like you have a clear destination. You're just... going. And as you bump into new stuff - things the model mashes together in ways you didn't expect - you change course. You explore. It's like the model is saying, "Hey, what about this?" and you're like, "Huh, never thought of that."
There's a buzz to it. A thrill in not knowing what's coming next. You're not trying to make some big, fancy project. You're just poking at your brain, seeing what comes out. It's like stretching a muscle you didn't know you had.
It's a new form of creative dialogue. The rapid-fire generation speed allows for a true back-and-forth, a conversation in visual language. You prompt, the model responds, sparking new ideas in your mind, leading to new prompts, and on it goes in a virtuous cycle. It's a form of "generative daydreaming." The boundaries between your initial concept and the model's output blur into one stream of continual discovery. You're not crafting a singular, static piece of media, but rather exploring possibilities. And it's joyful and fun.
This process taps into a part of our brains that craves novelty and surprise. It's not about the pressure to produce a film or a masterpiece. It's about flexing our creative muscles simply for the joy of the exercise. Like going to a gym for the mind, each session with the model leaves you invigorated, your imagination stretched in ways you didn't expect. When the tools are swift enough, you enter a flow state, a creative dialogue.
A form of play and discovery that's as rewarding as any final form. It's not about reaching a predetermined endpoint, it's more about reveling in the serendipitous exploration. -
Kenneth Stanley reposted
Cristóbal Valenzuela
@c_valenzuelab
I've been watching too many people immerse themselves for hours using Gen-3, and there's this pattern that keeps popping up. It's like this:
You start with some vague idea in your head. But as you play around, you end up in totally different places. It's weird - the twists and turns become more interesting than what you first thought of. It's not like you have a clear destination. You're just... going. And as you bump into new stuff - things the model mashes together in ways you didn't expect - you change course. You explore. It's like the model is saying, "Hey, what about this?" and you're like, "Huh, never thought of that."
There's a buzz to it. A thrill in not knowing what's coming next. You're not trying to make some big, fancy project. You're just poking at your brain, seeing what comes out. It's like stretching a muscle you didn't know you had.
It's a new form of creative dialogue. The rapid-fire generation speed allows for a true back-and-forth, a conversation in visual language. You prompt, the model responds, sparking new ideas in your mind, leading to new prompts, and on it goes in a virtuous cycle. It's a form of "generative daydreaming." The boundaries between your initial concept and the model's output blur into one stream of continual discovery. You're not crafting a singular, static piece of media, but rather exploring possibilities. And it's joyful and fun.
This process taps into a part of our brains that craves novelty and surprise. It's not about the pressure to produce a film or a masterpiece. It's about flexing our creative muscles simply for the joy of the exercise. Like going to a gym for the mind, each session with the model leaves you invigorated, your imagination stretched in ways you didn't expect. When the tools are swift enough, you enter a flow state, a creative dialogue.
A form of play and discovery that's as rewarding as any final form. It's not about reaching a predetermined endpoint, it's more about reveling in the serendipitous exploration. -
Kenneth Stanley reposted
samim
@samim
New Experiment: SerendipityLM - Interactive evolutionary exploration of generative design spaces with large language models.
Check out the extensive experiment report, incl. many generative art examples & link to open source repo: samim.io/studio/work/serendipityLM/
Find out more in this 🧵 -
Kenneth Stanley reposted
Elias Najarro
@EIiasNajarro
Making Music & Visuals with artificial life systems?
This year's @ALifeConf will host a Music/Visuals making hackathon. A space to explore how ALife systems can inspire visuals and music creation 🎶
More info: 2024.alife.org/hackathon.html @claire__aoi -
Kenneth Stanley
@kenneth0stanley
Great to see in-depth reflection on Why Greatness Cannot Be Planned - worth a listen!
Bruce Nielson
@bnielson01
I reflect on AI researcher @kenneth0stanley's assertion that setting specific, measurable goals may actually hinder discovery and innovation as argued in "Why Greatness Cannot Be Planned: The Myth of the Objective."
open.spotify.com/episode/2CUda5x4XsmGHgS2mxEc14 -
Kenneth Stanley
@kenneth0stanley
When LLMs fail at simple reasoning, many posit an inability to genuinely reason as the culprit. But there’s a deeper ingredient that underpins reasoning and many other faculties as well: representation. Before worrying about reasoning, worry about representation.
-
Kenneth Stanley reposted
Josh Bongard
@DoctorJosh
Want to help with #aidesignedbots?
1. Go to colab.research.google.com/drive/1h9MD64owkxvXD7Cp1BHn9Zmf… to design your bot.
2. Follow the instructions there.
3. Forward this tweet on. -
Kenneth Stanley
@kenneth0stanley
Much interesting space remains to explore for Quality Diversity and open-endedness in both Alife and AI.
Maxence Faldor
@maxencefaldor
Quality-Diversity + Lenia is a match made in heaven. ❤️
With @CULLYAntoine, we are excited to introduce Leniabreeder, a framework designed to automate the discovery of diverse artificial life. 🧬
🌐 Website: leniabreeder.github.io
📄 Paper: arxiv.org/abs/2406.04235
🧵 1/11 -
Kenneth Stanley reposted
Maxence Faldor
@maxencefaldor
Quality-Diversity + Lenia is a match made in heaven. ❤️
With @CULLYAntoine, we are excited to introduce Leniabreeder, a framework designed to automate the discovery of diverse artificial life. 🧬
🌐 Website: leniabreeder.github.io
📄 Paper: arxiv.org/abs/2406.04235
🧵 1/11 -
Kenneth Stanley
@kenneth0stanley
“Foundation model” is a poor metaphor for LLMs. A 3-year old is a foundation model. A 3-year old has a foundation of skills and basic concepts upon which almost anything can be built. An LLM is a multiple-personality mega-vac - it literally sucks up every concept and personality ever to be recorded. Then, instead of building on top of it like you would on top of a foundation, you excise tendencies and ideas that you don’t want through an alien lobotomy known as “fine-tuning.”
We don’t call a cathedral with three towers a “foundation” for a cathedral with one tower just because you can get to one by knocking two of them down.
Someday, when more genuine foundation models come out, they will be more efficient and impactful because we won’t need to build extra towers on our cathedrals only to intentionally knock them down later. -
Kenneth Stanley
@kenneth0stanley
The reason experts are so split on whether or not we're on an exponential path to AI is that intelligence is multi-dimensional. You can be both exponentially exploding/scaling in one dimension and stagnating in another. You can be both superhuman and subhuman at the same time. People tend to focus on the dimensions most salient to their inclinations, but the reality is that we can be simultaneously shooting up and stuck, and there is no good metaphor in the natural world for such an unnatural scenario (hence the "A" in AI). We can expect to see continued jaw-dropping improvement and hopeless stagnation all at once, and no precedent quite captures the strange mental chimera that results.
-
Kenneth Stanley reposted
Sakana AI
@SakanaAILabs
Can LLMs invent better ways to train LLMs?
At Sakana AI, we’re pioneering AI-driven methods to automate AI research and discovery. We’re excited to release DiscoPOP: a new SOTA preference optimization algorithm that was discovered and written by an LLM!
sakana.ai/llm-squared/
Our method leverages LLMs to propose and implement new preference optimization algorithms. We then train models with those algorithms and evaluate their performance, providing feedback to the LLM. By repeating this process for multiple generations in an evolutionary loop, the LLM discovers many highly-performant and novel preference optimization objectives!
Paper: arxiv.org/abs/2406.08414
GitHub: github.com/SakanaAI/DiscoPOP
Model: huggingface.co/SakanaAI/DiscoPOP-zephyr-7b-gemma
We proudly collaborated with the @UniOfOxford (@FLAIR_Ox) and @Cambridge_Uni (@MihaelaVDS) on this groundbreaking project. Looking ahead, we envision a future where AI-driven research reduces the need for extensive human intervention and computational resources. This will accelerate scientific discoveries and innovation, pushing the boundaries of what AI can achieve. -
Kenneth Stanley reposted
samim
@samim
It's fascinating that the Picbreeder experiment - A case study in collaborative evolutionary exploration of design space - which started in 2007 by @kenneth0stanley et al is still highly relevant and cutting edge today. wiki.santafe.edu/images/3/34/Stanley_innovation_workshop1… dl.acm.org/doi/abs/10.1162/EVCO_a_00030
-
Kenneth Stanley reposted
Ciaran
@ciaran_regan_
🥳 We released a new paper! 🥳
LLM-POET: Evolving Complex Environments using Large Language Models
A new approach in open-ended evolution using LLMs 🧵 -
Kenneth Stanley reposted
AIGuys
@RealAIGuys
Being honest/hard working in present and not missing the opportunity when it present itself gives the best of exploration and exploitation of the world. Thanks @kenneth0stanley for this beautiful book.
#ai #aiagent -
Kenneth Stanley
@kenneth0stanley
Great to see continuing deep thinking on the role of open-endedness in the current era of AI - congrats to @edwardfhughes and all the coauthors!
Edward Hughes
@edwardfhughes
📣 New paper! Open-Endedness is Essential for Artificial Superhuman Intelligence 🚀 (co-lead @MichaelD1729).
🔎 We propose a simple, formal definition for Open-Endedness to inspire rapid and safe progress towards ASI via Foundation Models.
📖 arxiv.org/abs/2406.04268
🧵[1/N] -
Kenneth Stanley reposted
AK
@_akhaliq
Open-Endedness is Essential for Artificial Superhuman Intelligence
In recent years there has been a tremendous surge in the general capabilities of AI systems, mainly fuelled by training foundation models on internetscale data. Nevertheless, the creation of openended, -
Kenneth Stanley reposted
Josh Bongard
@DoctorJosh
Looking forward to delivering a #cvpr24 keynote,
and getting to know this community better! -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Excited to introduce Intelligent Go-Explore: Foundation model (FM) agents have the potential to be invaluable, but struggle to learn hard-exploration tasks!
Our new algorithm drastically improves their exploration abilities via Go-Explore + FM intelligence. Led by @cong_ml🧵1/ -
Kenneth Stanley
@kenneth0stanley
One issue that I haven’t seen raised is that once you have AI at a genuinely human level you already have super-human capabilities because a group of many humans can achieve vastly beyond what a single human can alone (including in science and technology). So the more copies of that human-level AI you make, the more superhuman it gets as they work together, also undistracted by worldly needs like food and shelter. At a trillion copies, the aggregate can in principle do vastly more than all of human civilization combined. However, in practice it is still ultimately constrained by the need to perform experiments in the real world, and hence by the availability of finite resources to do those experiments. Yet the same constraint would also apply to a single super-intelligence - ultimately no matter how smart you are, you still must come up with hypotheses and try them out in the real world. In these ways, the capabilities of a population of “merely” human-level AIs is likely analogous to a single super-human. Which means we are closer to such an inflection than it may seem. (Not to mention the question of how powerful a population of even sub-human AIs can be versus one working on its own, which is probably the more salient and interesting issue in the near-term).
-
Kenneth Stanley
@kenneth0stanley
Nice summary of lessons from Picbreeder in this little clip of @tomowenmorgan !
Michael Garfield 💫
@michaelgarfield
This week on #futurefossils (now also on YouTube!), @tomowenmorgan and I talk about how to answer the call of our hearts — even and especially when it seems to require "hurling yourself into the abyss".
A superb discussion on faith and courage for everyone in a major transition. -
Kenneth Stanley
@kenneth0stanley
Very cool idea here!
Jeff Clune
@jeffclune
I am thrilled to introduce OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code. Led by @maxencefaldor and @jennyzhangzt, with @CULLYAntoine and myself. 🧵👇
-
Kenneth Stanley reposted
Rebecca Bellan
@RebeccaBellan
techcrunch.com/2024/05/26/how-mavens-ai-run-serendipity-n…
I got really into the tech/ philosophy behind former OpenAI researcher @kenneth0stanley's new social media platform Maven, which raised $2m seed funding last year from @ev and @sama and wants to make the internet less toxic. -
Kenneth Stanley
@kenneth0stanley
Thank you to @RebeccaBellan of @TechCrunch for the opportunity to share the vision behind Maven with her audience. Her thoughtful article is at techcrunch.com/2024/05/26/how-mavens-ai-run-serendipity-n…
-
Kenneth Stanley reposted
TechCrunch
@TechCrunch
How Maven’s AI-run ‘serendipity network’ can make social media interesting again tcrn.ch/44XW1tQ
-
Kenneth Stanley
@kenneth0stanley
What does it mean that an AI with the naivety yet aptitude of a 3-year old is harder to create than a simulacrum of an adult polymath?
Kenneth Stanley
@kenneth0stanley
Interesting to see a number of replies pointing to the lack of data or tokens from 3-year olds. But I think the awesomeness of the 3-year olds (or even 1-year olds) is that their faculties are superb because of how they will ingest the data in their future, not due to their output from the past (which such replies imply would allow us to replicate those faculties).
-
Kenneth Stanley reposted
John Borthwick
@Borthwick
Interview with @AnjanKatta about why he built Daylight -- at the end he talks about how he lives: phototropism: "a plant doesnt have a goal it has an attractor and it just gets there, trusting the process"
Find a direction, not a goal -- you have to trust the process, and that way, you find the path your path ... step by step cause greatness cant be planned.
h/t @kenneth0stanley
Jason Carman
@jasonjoyride
Interview with Founder of Daylight Computer @AnjanKatta
I really appreciated Anjan's unique views and approach in building a new computer company, there are a lot of wonderful insights here.This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
I had a great time talking to @trondau about Maven on @Futurized2 !
Futurized
@Futurized2
Social media can be better! @trondau interviews @kenneth0stanley about @HeyMaven_ , a serendipity network. www.futurized.org/serendipity-networks/ #socialmedia #serendipity
-
Kenneth Stanley
@kenneth0stanley
Maven is trending #1 on @ProductHunt ! You can help keep us on top with an upvote: www.producthunt.com/posts/maven-0f32d335-ea62-4654-b04c-b…
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
In Wired today: @kenneth0stanley's Maven (which I am involved in a bit via advising Ken on it). A nice explanation of its motivation and philosophy, which makes it an interesting alternative to mainstream social media. Via @SilverJacket Congrats Maven!
www.wired.com/story/maven-social-network-that-eliminates-… -
Kenneth Stanley reposted
Matthew Hutson
@SilverJacket
“Maven Is a New Social Network That Eliminates Followers—and Hopefully Stress”: My latest for @Wired.
Thanks, @kenneth0stanley, @BlasMoros, @jsecretan, @ev. www.wired.com/story/maven-social-network-that-eliminates-… -
Kenneth Stanley
@kenneth0stanley
Great article in Wired on Maven by Matt Hutson ( @SilverJacket )! Nice to see open-endedness and serendipity make an appearance in Wired!
WIRED
@WIRED
Twitter cofounder Ev Williams is backing a new social network called Maven, which trades likes and follows for algorithms designed to foster serendipity and deep discussion. wired.trib.al/6BGP8Cj
-
Kenneth Stanley
@kenneth0stanley
My most popular talk on Why Greatness Cannot Be Planned, from TTI/Vanguard, which many used as a reference, was unfortunately taken down recently when their YouTube channel was closed. So I obtained the video from them & reposted it to keep it available: youtu.be/VDuF4onPmuE
-
Kenneth Stanley reposted
daylight
@daylightco
COUNTDOWN TO LAUNCH
6 years in the making, 9 days to go
Take a deep breath 😮💨
Daylight is coming soon ✨ -
Kenneth Stanley reposted
Bentroen
@Bentroen
“The government, research labs, the corporate world, even your personal life — it's all objective-driven. But exploring interestingness requires you to accept that you don't know where you are going.”
Fantastic video by @SethBling and @kenneth0stanley: youtu.be/5zg_5hg8Ydo -
Kenneth Stanley reposted
Justin Chen
@ch3njus
Although not the same, Kolmogorov-Arnold Networks reminds me of the first machine learning project I worked on for my ml class in grad school. I modified @kenneth0stanley's NEAT algorithm to evolve neural networks that combined piecewise functions to play Mario, which was inspired by @SethBling.
Trip down memory lane. Can't believe that was eight years ago. I remember commandeering two rows of computers in the cs lab for a few days during finals week and spinning up ten terminals per computer (like 80 terminals) and bouncing back and forth to check each run so I could collect data for this report.
Final project report for CS542 from spring 2016:
arxiv.org/abs/1605.05216 -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Delighted to share that I've been promoted to Professor (aka “Full Professor”) A huge thanks to my wife, students, collaborators, colleagues, family, & friends for everything. It's been an exhilarating, wondrous, fascinating climb, and what a view! Now, which peak to climb next?
-
Kenneth Stanley
@kenneth0stanley
I had a great time speaking about Maven on the aptly-named Social Media Sucks Podcast. This was a fun and interesting discussion!
Kubbco
@KubbAndCo
Got a thing for AI, social media, or dreaming about where it's all heading? 🚀
You won't want to miss our latest podcast jam featuring the genius behind @HeyMaven_ , Dr. @kenneth0stanley
Plug in for a wild ride, here: www.kubbco.com/podcast/can-ai-fix-social-media-with-dr-ke… -
Kenneth Stanley reposted
Kubbco
@KubbAndCo
Got a thing for AI, social media, or dreaming about where it's all heading? 🚀
You won't want to miss our latest podcast jam featuring the genius behind @HeyMaven_ , Dr. @kenneth0stanley
Plug in for a wild ride, here: www.kubbco.com/podcast/can-ai-fix-social-media-with-dr-ke… -
Kenneth Stanley
@kenneth0stanley
It was a great pleasure to reminisce with @SethBling about his famous MarI/O NEAT experiment (seen by over 10 million people!) and the long and surprising road for me all the way from NEAT to the Maven social network today. Thanks Seth!!
SethBling
@SethBling
Almost 10 years ago I released MarI/O, a neural network that taught itself to play video games using Neuro Evolution of Augmenting Topologies (NEAT).
Check out my new interview with Ken Stanley, the author of NEAT:
youtu.be/5zg_5hg8Ydo -
Kenneth Stanley reposted
SethBling
@SethBling
Almost 10 years ago I released MarI/O, a neural network that taught itself to play video games using Neuro Evolution of Augmenting Topologies (NEAT).
Check out my new interview with Ken Stanley, the author of NEAT:
youtu.be/5zg_5hg8Ydo -
Kenneth Stanley reposted
Jon Ander Beracoechea
@jonberako
Following @kenneth0stanley recommendation I’ve just finished @erparker book “The Illusion of Innovation”. It’s a great book that builds on top of Kenneth’s ideas about how to innovate but focusing on a more corporate setting.
I have underlined many sections but I think this is my favorite quote:
“The reality is that the innovator’s dilemma is a very tricky problem to solve. To address it, companies must choose to accept some capital inefficiency in their operations. The optimal amount of capital inefficiency, contrary to popular belief, is not zero.”
Such a simple (and, for many, strange idea) -
Kenneth Stanley reposted
MeleeNeat
@MeleeNeat
Streaming current MAP Elites w/ hyperNEAT work on Super Mario Brothers.
www.youtube.com/watch?v=R1Let9fqOfw -
Kenneth Stanley reposted
Andrew Dai
@andrewdai99
ACES on code SFT
🌈 Teaming on QD-generated alignment data
A fun QD intro and timeline/papers: quality-diversity.github.io
Pioneered by many great researchers including @joelbot3000 @kenneth0stanley @jb_mouret @jeffclune @CULLYAntoine @snikolaidis19 and many more! -
Kenneth Stanley
@kenneth0stanley
Excited to be talking about Maven on the #socialmediasucks podcast taping Monday! See below to suggest questions in advance.
Kubbco
@KubbAndCo
We're excited to have the CEO of the new social media platform, Maven, @kenneth0stanley, joining us on the #socialmediasucks podcast on Monday! 🎙️
Have questions for us❓Drop'em in the comments and we'll see if we can get yours on the episode! 😆www.kubbco.com/podcast -
Kenneth Stanley
@kenneth0stanley
This book by my friend @erparker takes a thought-provoking perspective on innovation that also draws on ideas from Why Greatness Cannot Be Planned. It's worth a look!
Elliott Parker
@erparker
Tomorrow is release day for The Illusion of Innovation, a collection of lessons learned collaborating with entrepreneurs and changemakers for many years. I can't wait to hear what you think of it! (And it's not too late to pre-order: www.amazon.com/Illusion-Innovation-Efficiency-Unleash-Pro…) #illusionofinnovation
-
Kenneth Stanley
@kenneth0stanley
Optimization alone can't get you novelty or interestingness.
-
Kenneth Stanley reposted
Joel Lehman
@joelbot3000
Side project: Identifying Life-Changing Books with LLMs
tldr: Language models can analyze millions of book reviews to identify books that readers are most likely to say "changed their life."
blog (w/ complete list of 300 most 'life-changing' books): flourish.ing/identifying-life-changing-books-with-llms.html -
Kenneth Stanley reposted
CrossLabs
@crosslabstokyo
ICYMI: @kenneth0stanley's presentation "A Social Network Inspired by Open-Endedness" is available on our Youtube channel 🙌 www.youtube.com/live/73svHfR3eo8
-
Kenneth Stanley reposted
hardmaru
@hardmaru
New Paper Out.
Evolutionary Optimization of Model Merging Recipes
arxiv.org/abs/2403.13187
Our goal is not about training any particular individual foundation model. Instead, we think it makes more sense to create the machinery to automatically generate foundation models for us!
Sakana AI
@SakanaAILabs
Introducing Evolutionary Model Merge: A new approach bringing us closer to automating foundation model development. We use evolution to find great ways of combining open-source models, building new powerful foundation models with user-specified abilities!
sakana.ai/evolutionary-model-merge/ -
Kenneth Stanley reposted
Julian Togelius
@togelius
Should you let an LLM, such as ChatGPT, help you make your game? Will it add cool/fun features? Is the code any good? Will it be aesthetically coherent? We set out to find out in our new paper, "The Ink Splotch Effect: A Case Study on ChatGPT as a Co-Creative Game Designer".
-
Kenneth Stanley reposted
CrossLabs
@crosslabstokyo
The session with @kenneth0stanley will start at 8:30(JST), live on YT youtube.com/live/73svHfR3eo8
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Excited to share a project I have been working on at DeepMind. Congrats to the entire, wonderful SIMA team! I look forward to hearing what the community thinks of this advance.
Google DeepMind
@GoogleDeepMind
Introducing SIMA: the first generalist AI agent to follow natural-language instructions in a broad range of 3D virtual environments and video games. 🕹️
It can complete tasks similar to a human, and outperforms an agent trained in just one setting. 🧵 dpmd.ai/3TiYV7d -
Kenneth Stanley reposted
CrossLabs
@crosslabstokyo
We're thrilled to have @kenneth0stanley joining us this week to present 'A Social Network Inspired by Open-Endedness.'
Join us this Friday from 9am JST via YouTube Live 👉 www.youtube.com/live/73svHfR3eo8 #crossroads -
Kenneth Stanley reposted
CrossLabs
@crosslabstokyo
Save the date! Next Friday, @kenneth0stanley will join us on Cross Roads to talk about how studying open endedness in AI and ALife led him on an exciting adventure to create a new, innovative social network. Mark your calendars for March 15 @ 9AM JST. More details to come!
-
Kenneth Stanley reposted
Nikolaus Kriegeskorte
@KriegeskorteLab
A radically different social network driven by an inspiring vision...
Kenneth Stanley
@kenneth0stanley
Announcing Maven: We’ve created a new kind of social network - a serendipity network - that’s directly inspired by insights from open-endedness and Why Greatness Cannot Be Planned.
iPhone: apps.apple.com/app/maven-the-serendipity-network/id644960…
Android: play.google.com/store/apps/details?id=com.heymaven.maven
It's very different....
More 👉
On Maven, you follow interests instead of following people. The result is it’s no longer a popularity contest because you don’t need followers and we don’t count likes. The open-ended approach in effect provides an alternative to objective popularity-driven social media. And the resultant change in incentives means genuine, curiosity-driven conversations are more likely because getting attention stops being the primary concern.
If you’re familiar with ideas from open-endedness and quality diversity within AI and ML, Maven draws from these research areas to bring you a far broader diversity of views and serendipitous opportunities for engagement.
Everyone on Maven has an equal chance of engagement on day one because of the fact that you don’t need to gain followers. Maven effectively exposes everyone to posts that match their interests. So you’ll see more deep discussion and less click-bait and brand-building, because those things don’t accomplish anything on Maven.
It’s basically a large-scale experiment on open-endedness with humans in the loop, applied in the social domain.
(with amazing co-founders @jsecretan and @BlasMoros) -
Kenneth Stanley reposted
Brian Mosley
@Brian_Mosley_UK
Finally, a healthy alternative to social media, where you get to have quality conversations on topics that interest you.
No audience or group think, nobody to impress or follow, just stimulating conversation with fascinating people who find you.
Kenneth Stanley
@kenneth0stanley
Announcing Maven: We’ve created a new kind of social network - a serendipity network - that’s directly inspired by insights from open-endedness and Why Greatness Cannot Be Planned.
iPhone: apps.apple.com/app/maven-the-serendipity-network/id644960…
Android: play.google.com/store/apps/details?id=com.heymaven.maven
It's very different....
More 👉
On Maven, you follow interests instead of following people. The result is it’s no longer a popularity contest because you don’t need followers and we don’t count likes. The open-ended approach in effect provides an alternative to objective popularity-driven social media. And the resultant change in incentives means genuine, curiosity-driven conversations are more likely because getting attention stops being the primary concern.
If you’re familiar with ideas from open-endedness and quality diversity within AI and ML, Maven draws from these research areas to bring you a far broader diversity of views and serendipitous opportunities for engagement.
Everyone on Maven has an equal chance of engagement on day one because of the fact that you don’t need to gain followers. Maven effectively exposes everyone to posts that match their interests. So you’ll see more deep discussion and less click-bait and brand-building, because those things don’t accomplish anything on Maven.
It’s basically a large-scale experiment on open-endedness with humans in the loop, applied in the social domain.
(with amazing co-founders @jsecretan and @BlasMoros) -
Kenneth Stanley
@kenneth0stanley
More exciting work announced pushing progress forward in open-endedness (this time from @mitrma and colleagues) - third day in a row of such an announcement! The field is really amping up!
Michael Matthews
@mitrma
I’m excited to announce Craftax, a new benchmark for open-ended RL!
⚔️ Extends the popular Crafter benchmark with Nethack-like dungeons
⚡Implemented entirely in Jax, achieving speedups of over 100x
1/ -
Kenneth Stanley reposted
Mikayel Samvelyan
@_samvelyan
Introducing 🌈 Rainbow Teaming, a new method for generating diverse adversarial prompts for LLMs via LLMs
It's a versatile tool 🛠️ for diagnosing model vulnerabilities across domains and creating data to enhance robustness & safety 🦺
Co-lead w/ @sharathraparthy & @_andreilupu -
Kenneth Stanley
@kenneth0stanley
Yet another example of exciting new directions building on and extending ideas in open-endedness! Congrats to @_samvelyan and your collaborators!
Mikayel Samvelyan
@_samvelyan
Like Rainbow Teaming, we build on stepping stones (of ideas) generated by trailblazing visionaries like @kenneth0stanley @jeffclune @joelbot3000 (and many others!) and hope that ideas from open-endedness can further improve the safety of foundational models @EthanJPerez @janleike @yaringal @sleepinyourhat @JacobSteinhardt @jayelmnop @herbiebradley
-
Kenneth Stanley
@kenneth0stanley
Great concept and implementation from the Open-Endedness Team at @GoogleDeepMind . Open-endedness remains a critical way forward for AI! I'm thrilled to see it thriving!
Tim Rocktäschel
@_rockt
I am really excited to reveal what @GoogleDeepMind's Open Endedness Team has been up to 🚀. We introduce Genie 🧞, a foundation world model trained exclusively from Internet videos that can generate an endless variety of action-controllable 2D worlds given image prompts.
-
Kenneth Stanley
@kenneth0stanley
Thank you @MLStreetTalk and its fantastic host @ecsquendor for this chance to explain why my experiences after publishing Why Greatness Cannot Be Planned led to me start the Maven social network - an open-ended social network that for once isn't based on likes or follows.
Machine Learning Street Talk
@MLStreetTalk
Kenneth Stanley has launched a brand new social network called Maven based on the ideas from "Why Greatness Cannot Be Planned"! @kenneth0stanley @HeyMaven_
-
Kenneth Stanley reposted
TIAN Wei
@tianwei
One could hardly imagine how much #sora has become topic of the week in China since OpenAI made it known to the public. A good talk with @kenneth0stanley & @joelbot3000 on latest AI development and their book “Why Greatness Cannot Be Planned”. looking like an exciting Dragon Year
-
Kenneth Stanley reposted
Jenny Zhang
@jennyzhangzt
OMNI is accepted at #iclr2024! Updated paper has exciting new results in an embodied robotics domain, where AI learns new interesting tasks endlessly in an INFINITE task space. Open-endedness through Models of Human Notions of Interestingness
Website: www.jennyzhangzt.com/omni 🧵👇 -
Kenneth Stanley
@kenneth0stanley
The idea below is thought-provoking, but the deeper issue is not whether AI is trained to be servile vs. antagonist, but rather whether it is interesting. A one-dimensional stereotype of a human archetype isn’t deeply interesting or compelling either way. So what is?
Ian Arawjo
@IanArawjo
Most AIs are sycophants. What if we built antagonistic AI? 😈 Check out our working paper (w/ @_alicecai and @roboticwrestler ) wherein we argue that we should explore AI that are purposefully antagonistic towards users. No, seriously: (1/10) arxiv.org/abs/2402.07350
-
Kenneth Stanley reposted
Claudiu Leoveanu-Condrei
@leoveanv
Those who preached of open-endedness will be deeply rewarded over time.
(cc) @kenneth0stanley
www.the-scientist.com/news/novelty-activates-a-long-nonco… -
Kenneth Stanley
@kenneth0stanley
I asked ChatGPT to rate humanity on a scale of 1 to 10 and it gave us a 7:
“It's challenging to distill the complexity of humanity into a single numerical rating. However, considering our achievements, compassion, and potential for growth, I'd say around 7.” -
Kenneth Stanley reposted
Cristóbal Valenzuela
@c_valenzuelab
The most significant gap in AI research and art does not lie within the models themselves but rather in the approach to art. I have noticed a tendency to oversimplify the creative act. In a research setting, the goal is to control and measure variables, which leads to the assignment of constants for things that you can't measure. This logic swiftly leads to the interpretation of a "new algorithm for image generation" as a means to "be more creative." Attempting to standardize creativity as making pretty pictures simplifies the creative process to a convenient conclusion, which is why many AI tools seem designed more for engineers than for artists. It’s formulaic art. Creativity cannot be condensed into a formula for optimization. For a new technique for generating pixels on a screen to correlate with magnifying a creative act, you need four things: wonder, discovery, control, and feedback.
A tool dedicated to creation should facilitate wonder and discovery. The journey is just as critical as the outcome. Often, the creative process begins without a clear destination. Art is more than making pretty pictures; it's about conveying insights about the world, using tools as the medium. A proficient artist is perpetually engaged in discovery and wonder.
Effective wandering requires control over the tool and robust feedback mechanisms. You do something, you see something. That changes the direction and the interestingness of a space.
We need more artists doing research. -
Kenneth Stanley reposted
Art of the Problem
@Artoftheproblem
I just love @kenneth0stanley 's idea that "the only way find something is to not look for it" (in the context of evolutionary search & making the point of limitations of objective functions). It really nails why novelty matters in search problems
-
Kenneth Stanley
@kenneth0stanley
Thanks to everyone who has supported us in the last couple weeks. Just maybe it's possible with a little understanding of open-endedness to put some serendipity and divergence back into the internet.
Is there such a thing as a playground that isn't also a popularity contest?
Kenneth Stanley
@kenneth0stanley
Announcing Maven: We’ve created a new kind of social network - a serendipity network - that’s directly inspired by insights from open-endedness and Why Greatness Cannot Be Planned.
iPhone: apps.apple.com/app/maven-the-serendipity-network/id644960…
Android: play.google.com/store/apps/details?id=com.heymaven.maven
It's very different....
More 👉
On Maven, you follow interests instead of following people. The result is it’s no longer a popularity contest because you don’t need followers and we don’t count likes. The open-ended approach in effect provides an alternative to objective popularity-driven social media. And the resultant change in incentives means genuine, curiosity-driven conversations are more likely because getting attention stops being the primary concern.
If you’re familiar with ideas from open-endedness and quality diversity within AI and ML, Maven draws from these research areas to bring you a far broader diversity of views and serendipitous opportunities for engagement.
Everyone on Maven has an equal chance of engagement on day one because of the fact that you don’t need to gain followers. Maven effectively exposes everyone to posts that match their interests. So you’ll see more deep discussion and less click-bait and brand-building, because those things don’t accomplish anything on Maven.
It’s basically a large-scale experiment on open-endedness with humans in the loop, applied in the social domain.
(with amazing co-founders @jsecretan and @BlasMoros) -
Kenneth Stanley reposted
Mikayel Samvelyan
@_samvelyan
Uncovering vulnerabilities in multi-agent systems with the power of Open-Endedness!
Introducing MADRID: Multi-Agent Diagnostics for Robustness via Illuminated Diversity ⚽️
Paper: arxiv.org/abs/2401.13460
Site: sites.google.com/view/madrid-marl
Code: 🔜
Here's what it's all about: 🧵👇 -
Kenneth Stanley reposted
Blas
@BlasMoros
I'm thrilled to share more about Maven, a social network where you follow interests, not influencers
My cofounders are @kenneth0stanley and @jsecretan and we're backed by some amazing partners like @ev, @sama, @Lux_Capital, @mantisVC, @kaliouby, @beondeck, and others!
More 👉
On Maven, you follow interests, not influencers. This subtle change has deep implications. The game here is about building a genuine interest graph that helps you better discover new interests and like-minded people. It's like a self-organizing group chat on any topic you can imagine.
Why?
We use AI to add interest tags to every post and reply. This has a few awesome implications:
- Build personalized interest feeds: You only see things that are aligned with your interests and you can filter your feed based on these interests. Want to see a curated feed about AI art and discover others who follow it? On Maven you can.
- Discover awesome content whether posts or replies: Some of the best content normally gets buried as replies but because everything on Maven gets an interest tag and is surfaced to those who follow it, the discovery process is much better. This allows for the possibility of 'forever threads' and incentivizes deeper and more thoughtful conversations.
- Cross-pollinate ideas: If a post or reply touches on two different topics (e.g. video games and art), it will be shared with all users interested in games and also other users interested in art. In this way, communities are brought together around content. (Like a self-organizing group chat!)
- No followers needed: You don't need to amass a following to be heard. Simply follow your interests, engage with things you like, and serendipity will emerge as people with shared interests are automatically brought together.
@kenneth0stanley's announcement has some more details around the AI and serendipity angles: twitter.com/kenneth0stanley/status/1747674920933585220
Below is how you can get started:
iOS: apps.apple.com/app/maven-the-serendipity-network/id644960…
Android: play.google.com/store/apps/details?id=com.heymaven.maven
Kenneth Stanley
@kenneth0stanley
Announcing Maven: We’ve created a new kind of social network - a serendipity network - that’s directly inspired by insights from open-endedness and Why Greatness Cannot Be Planned.
iPhone: apps.apple.com/app/maven-the-serendipity-network/id644960…
Android: play.google.com/store/apps/details?id=com.heymaven.maven
It's very different....
More 👉
On Maven, you follow interests instead of following people. The result is it’s no longer a popularity contest because you don’t need followers and we don’t count likes. The open-ended approach in effect provides an alternative to objective popularity-driven social media. And the resultant change in incentives means genuine, curiosity-driven conversations are more likely because getting attention stops being the primary concern.
If you’re familiar with ideas from open-endedness and quality diversity within AI and ML, Maven draws from these research areas to bring you a far broader diversity of views and serendipitous opportunities for engagement.
Everyone on Maven has an equal chance of engagement on day one because of the fact that you don’t need to gain followers. Maven effectively exposes everyone to posts that match their interests. So you’ll see more deep discussion and less click-bait and brand-building, because those things don’t accomplish anything on Maven.
It’s basically a large-scale experiment on open-endedness with humans in the loop, applied in the social domain.
(with amazing co-founders @jsecretan and @BlasMoros) -
Kenneth Stanley reposted
GECCO 2026
@GeccoConf
Deadline for abstract submissions for full papers at #gecco2024 is Jan 25, 2024!!! Full paper it is 1 February 2024. Please follow the paper submission instructions in the following page: gecco-2024.sigevo.org/Paper-Submission-Instructions
-
Kenneth Stanley
@kenneth0stanley
Very interesting article with a great synthesis of work in open-endedness - shows that open-endedness can have major impact on fields outside conventional AI and ML!
Thomas Gorochowski
@chofski
🥳Latest from the lab is out now! We consider how open-endedness might offer a route to continual innovation in synthetic biological design. Sometimes it pays to take the path less well trod! Work led by @Michielstock and out in @ScienceAdvances. #synbio
doi.org/10.1126/sciadv.adi3621 -
Kenneth Stanley reposted
Bruce Nielson
@bnielson01
This was one of my favorite episodes!
I love the "Problem of Open-endedness." We need more people working on it.
With thanks to @kenneth0stanley, Leslie Valiant, and @DavidDeutschOxf for the source material I used here.
open.spotify.com/episode/2kbnAHtUxTZPOyzX4I5BGe -
Kenneth Stanley
@kenneth0stanley
Announcing Maven: We’ve created a new kind of social network - a serendipity network - that’s directly inspired by insights from open-endedness and Why Greatness Cannot Be Planned.
iPhone: apps.apple.com/app/maven-the-serendipity-network/id644960…
Android: play.google.com/store/apps/details?id=com.heymaven.maven
It's very different....
More 👉
On Maven, you follow interests instead of following people. The result is it’s no longer a popularity contest because you don’t need followers and we don’t count likes. The open-ended approach in effect provides an alternative to objective popularity-driven social media. And the resultant change in incentives means genuine, curiosity-driven conversations are more likely because getting attention stops being the primary concern.
If you’re familiar with ideas from open-endedness and quality diversity within AI and ML, Maven draws from these research areas to bring you a far broader diversity of views and serendipitous opportunities for engagement.
Everyone on Maven has an equal chance of engagement on day one because of the fact that you don’t need to gain followers. Maven effectively exposes everyone to posts that match their interests. So you’ll see more deep discussion and less click-bait and brand-building, because those things don’t accomplish anything on Maven.
It’s basically a large-scale experiment on open-endedness with humans in the loop, applied in the social domain.
(with amazing co-founders @jsecretan and @BlasMoros)
Kenneth Stanley
@kenneth0stanley
Maven is also available on the web: app.heymaven.com
-
Kenneth Stanley reposted
Sakana AI
@SakanaAILabs
Excited to announce our seed round!
We raised $30M to develop nature-inspired AI in Japan.
sakana.ai/seed-round -
Kenneth Stanley reposted
Josh Wolfe
@wolfejosh
Lux is thrilled to lead the $30M seed round of Japan's new leading AI group led @hardmaru and this incredible founding team of @SakanaAILabs
Takuya Akiba
@iwiwi
Sakana AIに入社しておりました!資金調達も大変上手く行ったようで、国内外からの高い期待を実感し改めて身が引き締まる思いです。素晴らしいメンバーと共に、おもろい研究を色々と計画してるので、乞うご期待です!
Sakana AI
@SakanaAILabs
Excited to announce our seed round!
We raised $30M to develop nature-inspired AI in Japan.
sakana.ai/seed-round -
Kenneth Stanley reposted
Joel Lehman
@joelbot3000
(1) I think AI safety is an important and philosophically deep research area.
(2) I think AI safety is likely pre-paradigmatic - that much of what we think we know is likely to be misleading and future research will overturn present formalisms.
(3) Given that, and how important many folks think AI safety is, most funding in AI safety seems (to me?) strangely conservative: Progress in this kind of research regime seems to require a lot of creativity and exploration, but I'm not aware of institutions / funding that support it? -
Kenneth Stanley reposted
Tim Rocktäschel
@_rockt
"It's really surreal. I do have to pinch myself, because I made these discoveries accidentally, they were serendipitous discoveries."
As all good discoveries should be.
www.bbc.co.uk/news/science-environment-67950749 -
Kenneth Stanley reposted
Tim Rocktäschel
@_rockt
In 2024, many of us will realize that what we have been trying to build all along are open-ended systems 🚀
It's no coincidence that @UCLCS will offer a new module on "Open-Endedness and General Intelligence" in the academic year 2024/25.
Mikayel Samvelyan
@_samvelyan
The surge in #openendedness research on arXiv marks a burgeoning interest in the field!
The ascent is largely propelled by the trailblazing contributions of visionaries like @kenneth0stanley, @jeffclune, and @joelbot3000, whose work continues to pave new pathways. -
Kenneth Stanley
@kenneth0stanley
Wow interesting—I had not realized before this chart what a phenomenon open-endedness is becoming within AI and ML. As it should be—intelligence without open-endedness is like flight without the ability to take off. This will increasingly matter! (Thanks @_samvelyan !)
Mikayel Samvelyan
@_samvelyan
The surge in #openendedness research on arXiv marks a burgeoning interest in the field!
The ascent is largely propelled by the trailblazing contributions of visionaries like @kenneth0stanley, @jeffclune, and @joelbot3000, whose work continues to pave new pathways. -
Kenneth Stanley
@kenneth0stanley
Something interesting to try: If you feel strongly about something--say some controversial topic where emotions are high--instead of thinking about the reasons your feelings are justified (which is the natural impulse), instead think about why your feelings are your feelings. In other words, the reasons you feel emotionally drawn to a view are not the same as the logical arguments that support your view - your emotions are not a function of a chain of logical implications. So then what *is* driving your feelings? The answer to that, if you can honestly find it, is more revelatory than all your clever arguments, because after all, it's the real reason you feel that way.
-
Kenneth Stanley reposted
Daniel Vaughan
@DBVaughan
Checkout this terrific interview with @kenneth0stanley. His work greatly resonates with me. I've struggled in big-tech where my research led to friction because it didn't have immediate 'impact'. Now I see it as providing potential stepping stones.
youtu.be/lhYGXYeMq_E -
Kenneth Stanley
@kenneth0stanley
The fact that a human-created architecture became ascendant before one from architecture search suggests that something has been wrong with the formulation of NAS, both in the formalization of the search space and in the conception of success, which is likely too focused on objective performance instead of on interestingness.
An intriguing question is what kind of NAS would have invented transformers on its own. While it didn’t happen that way, it’s possible to imagine a search process that on its own comes to view riffing on the idea of attention as *interesting*, but without too much concern about optimality, which would allow it to “novelty search” around this subspace without falling into deceptive local optima.
But then the question is, how would a search an algorithm develop an “interest?” Well, there’s plenty of research in that direction , so it’s at least conceivable.
And if that’s conceivable, then the next question is, what *else* is interesting, and without doubt there is a lot missing from current LLMs that would be quite interesting (e.g. to update weights continually based on live interactions without prediction, to learn from much less data, etc.) While NAS has so far run a step behind human inventors, that could change in the future, and the relative lack of investment in this possibility is also an opportunity to fill a vacuum.
Richard Socher
@RichardSocher
What happened to architecture search? It had so much potential to help AI self improve but afaics nobody is using a model found by it.
Maybe it'll get unlocked at another scale since it's just so expensive? -
Kenneth Stanley reposted
Andrew Dai
@andrewdai99
Looking for last-minute Xmas gift ideas?
💡All the copies of @kenneth0stanley and @joelbot3000’s book for friends & family - free accumulation of gifts via serendipity (from newfound knowledge), too!
🎄With greats, Advik and Chinaza, from @FormulaTrinity in Dublin!
Merry Xmas! -
Kenneth Stanley reposted
Chris Quinn
@cfsquinn
Listen to episode 148 of @shaneparrish's The Knowledge Project. @kenneth0stanley wonderfully critiques the rigidity of education that emphasizes pandering to objective measures instead of creating cultures that embrace "interestingness" in pursuit of better ways of educating.
-
Kenneth Stanley
@kenneth0stanley
Sounds like you might enjoy Why Greatness Cannot Be Planned.
Eliezer Yudkowsky ⏹️
@ESYudkowsky
Nobody called LLMs, as far as I know. One guy at OpenAI built GPT-1 while the others were working on deep RL, OpenAI Gym and such.
Technology in general and AI in particular is not that predictable! People thought at one point that defeating human grandmasters at chess would probably take AGI! And that's not because they were stupid, it's because humanity didn't always know what we know now.
The history of which AI technology gets developed first is the history of which problems turned out to be surprisingly easy to solve by some algorithm that wouldn't obviously go that far. The history of disappointments? Wrong guesses about which algorithms do solve things. It wasn't stupid to guess we'd have robotic cars by now. I didn't make that guess, but I didn't say on the Internet that the other guess was stupid, because I didn't know, and neither did you.
Do you think you know better? Then say it in advance or nobody will or should believe you.
Say today what AI cannot do in 2025 that others think it can, say today what AI will do in 2025 that no industry leader expects. Bet now in the prediction markets or forever hold your peace.
Endpoints can sometimes be predicted but exact trajectories are hard.
Think you can do better? Think you could have predicted LLMs in 2016, or 2003, unlike me? I don't believe you until you prove it with an advance prediction record, and neither should anyone else.
The people who are like "lol MIRI didn't predict LLMs in 2003" are ignorant of history, and specifically that OpenAI and Deepmind didn't predict LLMs in 2016.
The people who are like "LLMs are so unlike what MIRI predicted!" are making a very closely related mistake, the mistake of thinking that we'd be confident enough to strongly predict which AI problems would be easiest to solve. We worked on a more abstract level and picked concrete examples that didn't sound stupid in 2013, knowing we didn't know exactly what would be possible in 2023. But some people are not abstract thinkers and can only ever imagine that, if I gave an example of optimization involving a malfunctioning time machine, I must have imagined AI to be exactly like that; while of course, they imagine, the average OpenAI employee was expecting ChatGPT. -
Kenneth Stanley
@kenneth0stanley
Much discussion of AI's future extrapolates from current approaches, but things like pre-training and RLHF can become archaic surprisingly fast. One day there will be no "pre" and no "prompt," and enduring weight-level updates will happen for as long as "life" continues, just like with humans. So you have to understand, the idea of "values" that are baked into such a continually plastic system means something different. Try to extrapolate too far from what we have today, and you risk being deceived.
By the way, such a genuinely plastic system would be quite intriguing. "Context length" would no longer be a thing and we would have for the first time in effect a kind of "immortal learner" that can keep on updating itself forever and that never dies. We're clearly not there today but there is a vast space of systems yet to be imagined. When these new paradigms are realized, much of what we think about today will be moot, regarding safety or otherwise. -
Kenneth Stanley reposted
GECCO 2026
@GeccoConf
Call for papers:
Topics include genetic algorithms, genetic programming, ant colony optimization and swarm intelligence, complex systems, evolutionary combinatorial optimization and metaheuristics, evolutionary machine learning. The full list on the web gecco-2024.sigevo.org/Call-for-Papers -
Kenneth Stanley reposted
Mario Krenn
@MarioKrenn6240
Just finished "Why Greatness Cannot be Planned" by @joelbot3000 & @kenneth0stanley
Why hunting for Objectives is short-sighted and true surprising innovation can only come through open-endedness.
Seems to be a critical component for future AI Scientists. Highly recommended! -
Kenneth Stanley
@kenneth0stanley
For those who have confidence in the path to AGI because you believe you know its signs, it’s worth remembering what John Stuart Mill wrote in 1843: “There remains one a priori fallacy or natural prejudice, the most deeply-rooted, perhaps, of all which we have enumerated; one which not only reigned supreme in the ancient world, but still possesses almost undisputed dominion over many of the most cultivated minds … This is, that the conditions of a phenomenon must, or at least probably will, resemble the phenomenon itself.”
This word of caution speaks both to those who think we are surely on its path (because you see its likeness in the cutting edge) *and* to those who are sure we are headed wrongly (because you see stark differences). Surprise is the only rule in complex spaces and certitude is the most tantalizing mistake. -
Kenneth Stanley reposted
ALOE Workshop
@aloeworkshop
Q: Why is it an exciting time to do research in open-ended learning?
A: Because of all of you! 🫵 (well said, @jeffclune)
Thanks to all who took part in the ALOE Workshop. It truly felt like a special community today. See you next year! -
Kenneth Stanley reposted
Julian Togelius
@togelius
At the @aloeworkshop, @MasterMilkX presents our recent work on open-ended learning in a rich evolved environment with lots of diverse agents, and mixed-initiative AI-human creativity
-
Kenneth Stanley reposted
Garðar Ingvarsson
@gardaring
Excited to present our work on adapting quality-diversity algorithms to multi-agent learning at @aloeworkshop today at #neurips2023.
Stop by the poster session at 12:45-1:45 to learn more!
Paper: arxiv.org/abs/2311.01829 -
Kenneth Stanley reposted
Thomas Miconi
@ThomasMiconi
If you want to know how to generate an infinity of simple meta-Reinforcement Learning tasks (but rich enough to include standard domains like Harlow, bandits, 2-step, etc.), come to the ALOE workshop poster session tomorrow (Friday 1pm, Room 211-213).
neurips.cc/virtual/2023/79220 -
Kenneth Stanley reposted
ALOE Workshop
@aloeworkshop
We can't wait to have Antoine Cully @CULLYAntoine, professor at @imperialcollege, speak at ALOE this year. Antoine is a pioneer in the field of evolutionary robotics, and has contributed a wide range of new methods for quality-diversity optimization and real-world applications.
-
Kenneth Stanley reposted
Andrew Dai
@andrewdai99
We’ve updated QDAIF with raw data results, scripts, and improvements to results on baselines, automated archive expansion, an extra task domain, and more!
💾: github.com/qdaif/qdaif_scripts_data
Thread (1/6) 👇
Andrew Dai
@andrewdai99
How can LLMs automatically generate meaningfully diverse, high-quality text responses in creative domains (like generating stories and poems)?
Introducing Quality-Diversity through AI Feedback (QDAIF)!
🌐: qdaif.github.io
📜: arxiv.org/abs/2310.13032
👇 Thread (1/11) -
Kenneth Stanley
@kenneth0stanley
I’ve seen some talk recently about the virtues of hallucination, implying that it’s a “good” thing because it’s “creative” to make things up or simply what LMs are “trained to do.” I see my own work sometimes come up in these discussions since I’ve written a lot on creativity and open-endedness. So I want to give my two cents: An important aspect of creativity is the ability to know what already exists, which is how you can push towards novelty. If you are confused about what is real and not real, then it’s hard to know what’s novel.
So the less-discussed flipside of hallucination is the tendency to present old ideas as novel, fundamentally misunderstanding what it means to be creative. And indeed we see LMs often do just that when pushed to exhibit genuine innovation. Ask e.g. for innovative new recipes, ideas for new genres of music, or inventions to help with some existing problem, and often what you get back is something that either already exists or already has been proposed! (Note I am not saying that’s always what you get back, but neither do you always get hallucinations when asking questions of fact. But it happens too often.)
So false creativity and hallucination are two sides of the same coin. If you have one, you would tend to expect to see the other. In this sense, these are not good things, and the argument that hallucination is an asset “because creativity” is a misunderstanding. A firm grasp of reality is an ally of both authority and creativity, and indeed human authorities on subjects are also often creative.
But this point is not just to be cynical: it means that if we genuinely “solve” hallucination, creativity will come along for the ride. It also surfaces an interesting hypothesis about which approaches to solving hallucination are most deeply illuminating for progress in the field: if a “solution” only addresses one side of this two-sided coin (e.g. lowers hallucination without increasing creative quality) it is likely ultimately relatively superficial and not the genuinely transformational path that will lead to something spectacular in the field.
Kenneth Stanley
@kenneth0stanley
Woops that was the first time I edited a post and I should have read the warning more closely - didn’t realize a small edit would cut off all the quotes and comments already there! Sorry to @MinqiJiang @MLStreetTalk @c_valenzuelab @tobias_rees and others whose thoughtful comments got detached. Wish I could re-add them.
Kenneth Stanley
@kenneth0stanley
By the way, regarding the argument that by hallucinating LMs are simply doing what they were “trained to do”: they may or may not be trained to do it, but even if we assume they are, being trained to do something that is pathological does not make it good. If we are training them to hallucinate and (hand in hand with that) be falsely creative, all it implies is that we need to find a better way to train them. The only reason we train them to do something problematic is that we don’t yet know a better alternative, not that it’s some kind of clever, intentional sleight of hand.
-
Kenneth Stanley reposted
leonoel
@36Rleonoel
"Why greatness cannot be planned" is the best book I've read about AI so far
@kenneth0stanley @joelbot3000 -
Kenneth Stanley
@kenneth0stanley
For me, the question might be more whether I write any other book in my life! But thank you @BetterCallMedhi !
Mehdi (e/λ)
@BetterCallMedhi
If you read no other book in your life : this is the one.
-
Kenneth Stanley reposted
Mehdi (e/λ)
@BetterCallMedhi
If you read no other book in your life : this is the one.
-
Kenneth Stanley reposted
GECCO 2026
@GeccoConf
Call for Papers!
Exciting opportunity to contribute to GECCO 2024, a leading conference in genetic and evolutionary computation. Submit your research and be part of the innovation. Still open for submissions! #gecco2024 #callforpapers
gecco-2024.sigevo.org/Call-for-Papers -
Kenneth Stanley reposted
Brian Goldstone
@brian_goldstone
Now for the plea. Carole's doctors at Vanderbilt are convinced that this transplant can save her life. But @Cigna will need to reverse their decision—and quickly.
The company's CEO is David Cordani.
He can be reached at David.cordani@cigna.com -
Kenneth Stanley reposted
Brian Goldstone
@brian_goldstone
A thread about our depraved healthcare system, and a plea.
In 2021, my friend Carole was diagnosed with terminal stage four cancer. A single parent with two kids, she had just turned 43. In 2017, she had watched her brother Chris die of the same cancer. He too was in his 40s. -
Kenneth Stanley reposted
Pierre-Yves Oudeyer
@pyoudeyer
Take Flow Lenia, add random inflow and outflow of matter + ecological niches, what happens ? 🧐🔬
💡Spontaneous self-organization of an evolution process ? Generating exploring "agents"?This media couldn't be displayed. See it on the original post.
hardmaru
@hardmaru
Artificial lifeforms are super fascinating to watch.
These self-organizing, self-replicating, “lifeforms” emerged from a continuous time cellular automata system called Flow-Lenia.
Lenia is a family of CAs generalizing Conway’s Game of Life to continuous space, time and states. -
Kenneth Stanley
@kenneth0stanley
A reminder of how the like-causes-like fallacy (which leads to the objective paradox) touches so many of our ambitions.
clem 🤗
@ClementDelangue
Objectives and goals as false compasses and deceiving stepping stones. Fun to apply to companies: you don’t create a big company by increasingly looking like a big company
-
Kenneth Stanley reposted
Miko | Ascendance.☁️
@AscendanceCloud
The best path to reach your goal is never pointed directly towards the goal. The landscape of the game of life is much more complex than that. 🚞
Learn to nagive it. 🧭
#themythoftheobjective @kenneth0stanley @joelbot3000 -
Kenneth Stanley reposted
Andrew Dai
@andrewdai99
How can LLMs automatically generate meaningfully diverse, high-quality text responses in creative domains (like generating stories and poems)?
Introducing Quality-Diversity through AI Feedback (QDAIF)!
🌐: qdaif.github.io
📜: arxiv.org/abs/2310.13032
👇 Thread (1/11) -
Kenneth Stanley reposted
hardmaru
@hardmaru
Artificial lifeforms are super fascinating to watch.
These self-organizing, self-replicating, “lifeforms” emerged from a continuous time cellular automata system called Flow-Lenia.
Lenia is a family of CAs generalizing Conway’s Game of Life to continuous space, time and states. -
Kenneth Stanley reposted
Tom Sawada
@tsawada_ml
Quality-Diversity through AI Feedback
abs: arxiv.org/abs/2310.13032
pdf: arxiv.org/pdf/2310.13032.pdf
website: qdaif.github.io
- Quality-Diversity through AI Feedback (QDAIF): framework in which an evolutionary algorithm applies LMs to both generate variation and evaluate the quality and diversity of candidate text -
Kenneth Stanley reposted
Carlos De la Guardia
@dela3499
What’s a good way to spend a couple hours with someone?
How about evolving drawings together?
That’s what @poetengineer__ and I did on our long flight, inspired by evolutionary experiments like:
1. Biomorphs by @RichardDawkins
2. Picbreeder by @kenneth0stanley -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Today our research is on the homepage of the New York Times, covering VPT and implications for AI like it (eg @jluan's @AdeptAILabs), & great work by @DrJimFan & @AnimaAnandkumar).As a fun bonus, they also included my children's doodles! Thanks @CadeMetz!
www.nytimes.com/2023/10/16/technology/ai-agents-workers-r… -
Kenneth Stanley
@kenneth0stanley
Not a bad choice!
clem 🤗
@ClementDelangue
Reading of the week!
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
After a conversation with @joelbot3000 & @kenneth0stanley we concluded there’s an important AI safety point deserving broader discussion: In short, any mandatory “nutrition label” for foundation models needs to go well beyond just disclosures on training data... 🧵1/
-
Kenneth Stanley reposted
Jenny Zhang
@jennyzhangzt
Update on OMNI (Open-endedness via Models of human Notions of Interestingness). We tried it on a new, off-the-shelf domain (BabyAI), and OMNI outperforms baselines there too! Website: www.jennyzhangzt.com/omni 1/
-
Kenneth Stanley reposted
Josh Bongard
@DoctorJosh
For 29 years, AI-driven robot design has required weeks on a supercomputer.
Today, in #pnas, we report how modern AI can design robots in 30 seconds, on a laptop.
Joint work w/ David Matthews, Andrew Spielberg, Daniela Rus, and Sam Kriegman (@Kriegmerica) -
Kenneth Stanley reposted
William Latham
@WilliamLatham1
Scientific American, 30 years ago this month. Feature on Creative Evolution covering the work of Karl Sims, Michael Tolson and William Latham
-
Kenneth Stanley reposted
Chrisantha Fernando
@chrisantha_f
🌱 Introducing Promptbreeder: LLMs evolve their own prompts through self-referential self-improvement! Paper: arxiv.org/abs/2309.16797 #promptengineering #llm #ai #ml #promptbreeder
-
Kenneth Stanley
@kenneth0stanley
Just another example of the Myth of the Objective. Maximization does not lead to greatness.
Michael Sentef
@sentefmi
Just a quick reminder that you can win a Nobel with an h-index = 29 (according to the usually inflated google scholar). Quantitative measures of scientific output are not the way to go to a bright future for humanity.
-
Kenneth Stanley reposted
Tim Rocktäschel
@_rockt
Fun fact: the name Promptbreeder is a tribute to @kenneth0stanley's Picbreeder (ieeexplore.ieee.org/document/6793948), an open-ended human-in-the-loop system that evolves increasingly interesting images. While Picbreeder explores the space of images, Promptbreeder explores the space of prompts and does so without humans in the loop.
We also take inspiration from a range of recent LLM x Open-Endedness papers, in particular @joelbot3000's LMX (arxiv.org/abs/2302.12170), @guanzhi_wang's @DrJimFan's & @AnimaAnandkumar's Voyager (arxiv.org/abs/2305.16291), and @jennyzhangzt's & @jeffclune's OMNI (arxiv.org/abs/2306.01711). -
Kenneth Stanley reposted
Kenneth Stanley
@kenneth0stanley
By the way, the fact that it's hard for many people to even conceive of making choices that do not maximize something reveals a profound cultural undercurrent in our society. Optimization is so worshipped that it is inconceivable to many that anything else is even possible.
-
Kenneth Stanley
@kenneth0stanley
Though it does some optimization, natural evolution is not primarily an optimization algorithm. It's unfortunate that classical genetic algorithms created a misapprehension among computer scientists that evolution is best abstracted as optimization. So what is evolution? 1/2
Yann LeCun
@ylecun
I'd say *optimization* is the only way to obtain complexity from primeval simplicity.
There are many ways to optimize .
Darwinian evolution (mutate and select among a population) is just one particularly simple and *inefficient* way to perform zeroth-order optimization.
But there are many, many other methods, including 1st order (gradient-based) methods that are way more efficient whenever they are applicable.
Kenneth Stanley
@kenneth0stanley
Evolution is actually a set of entangled search processes that supports multiple abstractions. Among them: quality diversity, novelty search, competition escape, minimal criterion search, and the "spilled milk" abstraction discussed in Why Greatness Cannot Be Planned. 2/2
Post not available here, view on X
Kenneth Stanley
@kenneth0stanley
By the way, the fact that it's hard for many people to even conceive of making choices that do not maximize something reveals a profound cultural undercurrent in our society. Optimization is so worshipped that it is inconceivable to many that anything else is even possible.
-
Kenneth Stanley reposted
Mehdi (e/λ)
@BetterCallMedhi
Je viens de terminer la relecture de ce chef d’œuvre encore trop méconnu et que je conseille vivement pour naviguer dans le chaos et in fine l’inconnu.
Construisant des boîtes deeptech, ce livre a permis de poser des mots et des analyses précises sur ce que je ressentais au plus profond de moi à mes débuts.
Ce livre est véritablement un game-changer à une époque où les tendances et les comportements mimétiques triomphent en profitant de l’anéantissement totale de la curiosité innée et de notre véritable esprit d’explorateur :
Lorsque l’on construit quelque chose, on a tout de suite tendance à se baser sur des réussites, modèles ou méthodes connus et à essayer de reproduire tous ces process établis de manière générale et quasi-automatique quoi que l’on puisse bâtir.
Ces références permettent de conserver un confort intellectuel comme un bouclier contre les menaces du désordre et de l’inconnu et donc de minimiser les risques.
En réalité ces supports vérifiés plus ou moins rassurant brident totalement les percées singulières et découvertes fantastiques que nous pourrions réaliser.
En effet, ce genre d’attitude conformiste contribue à créer un frein quand vous prenez conscience que vous ne pouvez pas construire une boîte significative sans une culture profonde portée sur la serendipité à son maximum.
En d’autres termes, vous ne pouvez pas par exemple construire une boîte deeptech comme si vous construisiez une boîte software avec les mêmes process établis depuis plus de 20 ans tels que le business plan, la roadmap, le MVP, le PMF etc.
Or, l’erreur que font beaucoup d’entrepreneurs français (surtout ceux originaires des grandes écoles totalement formatés), c’est justement d’adopter un comportement trop souvent mimétique en reprenant les mêmes méthodologies qui ont fonctionné à un moment donné pour les entreprises software (reprises par les écoles de commerce) et de les appliquer pour des projets deeptech qui demandent avant tout un esprit d’explorateur à la Jules Verne.
En définitif, avec ce livre vous comprendrez que les grandes réalisations deviennent moins probables lorsqu'on en fait des objectifs précis.
À contre courant de la pensée des grandes écoles et des investisseurs, vous comprendrez aussi que le meilleure façon d'atteindre la grandeur, le chemin le plus véritable vers la découverte et la réalisation d'une ambition sans limites est en réalité de n'avoir aucun objectif du tout. -
Kenneth Stanley
@kenneth0stanley
It’s the right time for someone to do this! Congrats @hardmaru , glad it will be you (as well as @YesThisIsLion )!
hardmaru
@hardmaru
Personal Announcement! I’m launching @SakanaAILabs together with my friend, Llion Jones (@YesThisIsLion).
sakana.ai is a new R&D-focused company based in Tokyo, Japan.
We’re on a quest to create a new kind of foundation model based on nature-inspired intelligence! -
Kenneth Stanley reposted
Eleni Nisioti
@nisioti_eleni
In its warm nest
The sparrow is happy;
Niche construction!
Excited to be in Sapporo, Japan where I am practising my haiku skills and presenting our work at @ALifeConf with @Clement_MF_ on an evolutionary model of niche construction arxiv.org/abs/2305.09369
Quick summary below -
Kenneth Stanley
@kenneth0stanley
There are many ways to research open-endedness. Artificial life (alife) is still full of possibilities.
Ettore Randazzo
@RandazzoEttore
Finally ready to share Biomaker CA, a Biome Maker project using Neural Cellular Automata. w/ @zzznah See live article google-research.github.io/self-organising-systems/2023/bi… (with lots of videos) and arxiv paper arxiv.org/abs/2307.09320. 1/N
-
Kenneth Stanley
@kenneth0stanley
Great to have an open source version of ELM (Evolution though Large Models) available!
-
Kenneth Stanley reposted
Matt Fontaine
@tehqin17
I'm excited to announce my work with @snikolaidis19 on Covariance Matrix Adaptation MAP-Annealing has been nominated for best paper in the EML track @GeccoConf 2023!
abs: arxiv.org/abs/2205.10752
pdf: arxiv.org/pdf/2205.10752.pdf
code: github.com/icaros-usc/cma_mae -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Introducing: First-Explore, then Exploit: Meta-Learning Intelligent Exploration. Led by @BenNorman451 arxiv.org/abs/2307.02276
Humans are masters at exploring. Unlike RL, we do not explore by trying to maximize reward (with noise), but instead explore to gain information! 1/8 -
Kenneth Stanley
@kenneth0stanley
Interesting tidbit from @sama on the role of Why Greatness Cannot Be Planned in the history of OpenAI. (thanks to @actualMahmoud for highlighting)
-
Kenneth Stanley
@kenneth0stanley
Happy to know my friend @jeffclune has a voice in this important discussion!
Jeff Clune
@jeffclune
It was an honor to meet Senate Majority Leader Chuck Schumer @SenSchumer today to discuss how to safely innovate in AI to reap the massive benefits while minimizing risks & downsides, including the best path forward on regulation:A video of his impressively informed talk is below
-
Kenneth Stanley reposted
Tim Rocktäschel
@_rockt
The Open-Endedness Team at Google @DeepMind London is looking for a Research Engineer to join us. If you are excited about scaling up ambitious research, pushing state-of-the-art distributed compute and ML infrastructure to its limits, then apply 👇
boards.greenhouse.io/deepmind/jobs/5121254 -
Kenneth Stanley
@kenneth0stanley
Great idea to have this list of works on open-endedness!
Jenny Zhang
@jennyzhangzt
Fascinated by the idea of enabling AI to learn new, interesting behaviors forever? Started an awesome list for open-ended AI: github.com/jennyzzt/awesome-open-ended. Make a pull request to contribute relevant works! 😎
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Excited to share OMNI: Open-endedness via Models of human Notions of Interestingness. Lead: @jennyzhangzt arxiv.org/abs/2306.01711 Open-ended learning requires a vast space of possible tasks, but search thus can get lost. Agents must focus only on *interesting* tasks, but how?🧵1/
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Introducing Thought Cloning: AI agents learn to *think* & act like humans by imitating the thoughts & actions of humans thinking out loud while acting, enhancing performance, efficiency, generalization, AI Safety & Interpretability. Led by @shengranhu
arxiv.org/abs/2306.00323 1/5 -
Kenneth Stanley reposted
-
Kenneth Stanley reposted
TIAN Wei
@tianwei
Open-endedness v.s. Goals... GREAT fun talking to @kenneth0stanley @joelbot3000 two former Open AI scientists on GPT & AI's future with human. We can try to be "philosophers" while energetically embracing this round of tech revolution !
www.youtube.com/watch?v=OdxvIhzZydo&list=PLt-M8o1W_GdQ3OM… -
Kenneth Stanley reposted
Freethink
@freethinkmedia
"What does talent or skill mean anymore in such a world? Talent and skill are falling to computers, and we haven’t seen that before." @kenneth0stanley
explores this new dynamic in our special issue. Read more here: www.freethink.com/robots-ai/idea-amplification-is-the-rea… -
Kenneth Stanley
@kenneth0stanley
Thanks to @freethinkmedia for the opportunity to share some of my current thoughts on AI for creativity: www.freethink.com/robots-ai/idea-amplification-is-the-rea…
-
Kenneth Stanley
@kenneth0stanley
The question is not whether LLMs are intelligent but rather whether they are a stepping stone along one of many paths. Are our worm-like ancestors more “intelligent” than LLMs? Arguments about stepping stones are far harder, hence less amenable to certitude.
-
Kenneth Stanley reposted
Logan
@ChipkinLogan
Updated honorary guest speakers for Rat Fest:
Friday - @kenneth0stanley, author of Why Greatness Cannot Be Planned
Saturday - @DorianKBandy, author of Mozart the Performer
Sunday (q & a) - @DavidDeutschOxf, author of The Fabric of Reality and The Beginning of Infinity -
Kenneth Stanley reposted
Matt Fontaine
@tehqin17
I'm really excited for this workshop! We hope to bring together the PCG, scenario generation, and open-ended co-evolution communities, as they are all interested in environment generation in different contexts, to exchange ideas and promote generalization in robotics.
Varun Bhatt
@vbhatt_cs
We are excited to announce the first workshop on Environment Generation for Generalizable Robots (EGG) at #rss2023 (sites.google.com/usc.edu/egg-rss-2023)! Consider submitting if you are working in any area relevant to environment generation for robotics. Submissions due on May 17, 2023, AoE.
-
Kenneth Stanley
@kenneth0stanley
Too kind @wolfejosh , thank you, and super-excited for what we're going to do!
Josh Wolfe
@wolfejosh
Ken is absolute brilliant technological + computer science embodiment of the virtue of 'randomness & optionality'...
Kenneth Stanley
@kenneth0stanley
As some of you know, I've been "doing something new" that I hope to share soon! It happens that being in the Lux Capital family is part of that journey, and I couldn't be happier to congratulate Lux on the launch of their latest fund - Lux ventures 8 www.luxcapital.com/lux8
-
Kenneth Stanley
@kenneth0stanley
As some of you know, I've been "doing something new" that I hope to share soon! It happens that being in the Lux Capital family is part of that journey, and I couldn't be happier to congratulate Lux on the launch of their latest fund - Lux ventures 8 www.luxcapital.com/lux8
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
My AI-Generating Algorithms paper described "Darwin-complete" environment search spaces: those that can represent any environment. I suggested one then: neural network world models. Another is code (e.g. LM-generated code for environments/simulators/games) arxiv.org/abs/1905.10985
-
Kenneth Stanley
@kenneth0stanley
Speaking of AI risks, if I gave a cutting-edge LLM (say with visual inputs too) an email address, access to a browser, a place to store notes, $1M in the bank with a credit card, and told it to “improve the human condition” in a continual loop, what would it do?
-
Kenneth Stanley reposted
Authored by AI
@Authoredby_AI
Episode 3 has a different structure than our other episodes: we use clips from several interviews with AI experts to create a kind of intro course to "AI that writes". Give it a listen! www.authoredby.ai/ai-that-writes-101/
-
Kenneth Stanley reposted
Logan
@ChipkinLogan
Honorary guest speakers for Rat Fest:
Friday - Kenneth Stanley @kenneth0stanley, author of "Why Greatness Cannot Be Planned: The Myth of the Objective"
Saturday - Dorian Bandy @DorianKBandy, author of forthcoming "Mozart the Performer"
Tickets: www.eventbrite.com/e/the-rat-fest-tickets-387125692367 -
Kenneth Stanley
@kenneth0stanley
For current approaches to GPT/LLMs and chatbots, what is important for progress is not what they don’t do well, but rather what they can *never* do well and why that is, as well as which “nevers” will require the most fundamental architectural or training overhauls.
-
Kenneth Stanley
@kenneth0stanley
This is a very unique and interesting program that I've visited before - well worth considering:
ToftH
@tofth__
A call for those struck by the experience that the philosophical questions their work generates cannot be addressed in terms of their current discipline. People who feel constrained by the limits of what is and who wish to invent new realities. Apply now tofth.org/page/apply
-
Kenneth Stanley reposted
Joel Lehman
@joelbot3000
Fun paper in the "Evolution through Large Models" spirit, Elliot Meyerson had great original insight this behind paper. Shows relationship b/t EDAs and few-shot prompting in LMs -- few-shot prompting can be viewed in effect as few-shot EDA (estimation of distribution algorithm)
-
Kenneth Stanley reposted
Joel Lehman
@joelbot3000
"Machine Love" - new paper w/ LM expts: Leaving to humans what is human (emotions/relationships/autonomy), is there any conception of love fitting for machines to embody? One idea: to provide unconditional support for humans to pursue their own growth.
arxiv.org/pdf/2302.09248.pdf -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
We also combine MarioGPT (as a mutation operator) with novelty search, NS-MarioGPT, allowing us to create new levels in an open-ended fashion. Different but related to the great work by @joelbot3000, @kenneth0stanley et al:
Joel Lehman
@joelbot3000
“Evolution through Large Models” – new paper from our team at OpenAI. Step towards evolutionary algos that continually invent and improve at inventing: Large models can suggest (+ improve at making) meaningful mutations to code. Paper: arxiv.org/abs/2206.08896 1/4
-
Kenneth Stanley reposted
Julian Togelius
@togelius
Large language levels can do... lots of things. But can they generate game levels? Playable game levels, where puzzles are solvable? Two papers announced today address this question. The first is by our team at @NYUGameLab - read on for more:
-
Kenneth Stanley
@kenneth0stanley
The solutions to almost all great puzzles (e.g. AGI) are counter-intuitive. Otherwise, they would be solved! So we should be suspicious when a solution is offered that sounds too intuitive. Reality is a radical!
-
Kenneth Stanley
@kenneth0stanley
Thanks to @peterjscott and the AI and You podcast for @joelbot3000 and my first opportunity to be interviewed together on the same show, focusing with great discussion topics mainly on Why Greatness Cannot Be Planned! aiandyou.net/e/135-guests-kenneth-stanley-and-joel-lehman…
-
Kenneth Stanley reposted
David Gasca
@gasca
"I have seen the greatest minds of my generation driven mad by endless planning sessions"...
I wrote up a post on how to move beyond "objectives" and be a treasure hunter
mysticalsilicon.substack.com/p/treasure-hunting -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Starting to hire PhDs and postdocs on my GROW-AI #erccog project: candidate.hr-manager.net/ApplicationInit.aspx?cid=119&Pro…
Application deadline: February 1st, 2023.
Join us and get in touch if interested in how we can grow neural networks for more general agents! Aiming to build a diverse team. Please share.
Sebastian Risi
@risi1979
Very grateful and honored to have been awarded an ERC Consolidator Grant of ~2 million Euro for my project GROW-AI: Growing Machines Capable of Rapid Learning in Unknown Environments. #erccog
Thanks to the @ERC_Research, my lab, and family for their support! -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
This work by @FerreiraFabioDE @FrankRHutter seems very cool! It further extends generative teaching networks (GTNs) for RL. Congrats to the team. Could help with auto-generating & grounding learning environments (Pillar 3 of AI-GAs) cc @kenneth0stanley @joelbot3000 @felipesuch
Frank Hutter
@FrankRHutter
We #metalearn proxy MDPs called "synthetic environments" for RL: given an RL environment E, you can train a neural net to replace E‘s original MDP & train policies for the original E more efficiently! openreview.net/pdf?id=C1_esHN6AVn, with @FerreiraFabioDE, Thomas N., @AndreasSalinger
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Love this edit! This video short sums up a quest I have been on for my entire scientific career!
(along with @kenneth0stanley @joelbot3000 @jb_mouret and others)
The TWIML AI Podcast
@twimlai
Great pull by @gssgek, here's the full quote for context: youtube.com/shorts/dp07w88YzTw
-
Kenneth Stanley reposted
Bryan Lim
@bryanlimwt
Interested in how to train diverse population of agents more efficiently? Check our our new paper being presented at the @NeurIPSConf DeepRL workshop today!
Paper: arxiv.org/abs/2211.12610
This is work done with amazing collaborators @MFlageat (Project Co-lead) and @acully. -
Kenneth Stanley
@kenneth0stanley
I asked ChatGPT to come up with a new programming language where you just write code in plain English. It wrote code for a couple games as examples. When will this be available? (I'm not sure the principle behind the color coding in this language!)
-
Kenneth Stanley
@kenneth0stanley
Great work @carperai ! Evolution through Large Models (ELM) in open source!
-
Kenneth Stanley reposted
Julian Togelius
@togelius
New @MLStreetTalk podcast (and video) featuring me and @kenneth0stanley out now! It was recorded a few months ago, but I remember it as a very interesting conversation. Hope you find it interesting to listen to as well! (I may have one or two spicy opinions in there, IIRC.)
Machine Learning Street Talk
@MLStreetTalk
New show with Prof. Julian Togelius @togelius (NYU) and Prof. Ken Stanley @kenneth0stanley we discuss open-endedness, AGI, game AI and reinforcement learning. youtu.be/iqkkkA9dtY0
-
Kenneth Stanley reposted
Machine Learning Street Talk
@MLStreetTalk
New show with Prof. Julian Togelius @togelius (NYU) and Prof. Ken Stanley @kenneth0stanley we discuss open-endedness, AGI, game AI and reinforcement learning. youtu.be/iqkkkA9dtY0
-
Kenneth Stanley reposted
Bert Chan
@BertChakovsky
Inria 🇫🇷 master internship 2023 on Lenia x machine learning research is open, apply now!
Based on past projects, we will continue to train/evolve Lenia creatures for energy seeking, interactions, memory, learning, open-endedness, etc. 1/4
More info: flowers.inria.fr/jobs/ -
Kenneth Stanley
@kenneth0stanley
Mostly missing from current ML discourse: the order in which things are learned profoundly affects their ultimate representation, & representation is the true heart of “understanding.” But we just shovel in the data like order doesn’t matter. No AGI without addressing this. 1/2
Kenneth Stanley
@kenneth0stanley
Worth considering: Open-ended systems (which include infants and children) learn in a different order than objective-driven systems. 2/2
Kenneth Stanley
@kenneth0stanley
A couple people have asked whether there is any research to point to on open-ended vs. objective-derived representations. Like I said, it's mostly missing, but I have an old paper from 2011 with @fletchlingArrow with a unique perspective on it: eplex.cs.ucf.edu/papers/woolley_gecco11.pdf
-
Kenneth Stanley
@kenneth0stanley
Looking forward to speaking at this event! Registration link is below.
Alloy Partners
@alloy_partners
Excited to welcome @kenneth0stanley for our next Satellite Series on December 2!
Full details + register now: www.eventbrite.com/e/satellite-series-ken-stanley-plannin… -
Kenneth Stanley reposted
Julian Togelius
@togelius
I've been looking forward to announce this! Yes, we are growing the company and will be hiring. It's all about game-playing bots to improve testing and other aspects of game development.
modl.ai
@modl_ai
We are thrilled to announce that we’ve raised $8.4 million to redefine the game development process. We aim to make it more efficient and enjoyable.
We’re eager to get the product to you soon; this funding will help us get there faster. venturebeat.com/games/modl-ai-seriesa-ai-bot-qa-testing-g… -
Kenneth Stanley reposted
Samuel Arbesman
@arbesman
Emergent Microcosms 🌎💻
Time to think about snippets of computer code that can generate complex + delightful virtual worlds.
This playful approach to emergence—algorithms that can unfurl entire virtual universes—are essentially “Emergent Microcosms”
🧵
1/ -
Kenneth Stanley reposted
hardmaru
@hardmaru
“Hypernetworks” have taken on a new life in Stable Diffusion models.
-
Kenneth Stanley
@kenneth0stanley
This is a really nice video about Picbreeder-like evolution of art with modern tools, congrats @max_romana ! Also check out @Artbreeder as well for another great option
Max | Emergent Garden
@max_romana
New video! Using #midjourney and #stablediffusion to evolve Ai Art! www.youtube.com/watch?v=K8TG0ZwYu7Y
-
Kenneth Stanley reposted
Joseph Suarez 🐡
@jsuarez
Proposing a new Neural MMO competition on open-endedness and task-conditional RL! Attend the @carperai @StabilityAI talk + panel discussion with @togelius @jeffclune @robertarail @joelbot3000 Friday Oct 28th, 1:00 PM ET
Event: discord.gg/KhJRCQ8R?event=1034122655604940870
Doc: www.overleaf.com/read/vqgrccffsccr -
Kenneth Stanley reposted
David Sweet
@phinance99
Saturday fun: Making stable diffusion pumpkins with novelty search
@kenneth0stanley -
Kenneth Stanley reposted
Kanjun 🐙
@kanjun
Today, AI systems can create stunning art & beat humans at chess & Go. But they can't do things a 3-year-old can do. Why?
We're launching @genintelligent & open sourcing our research environment, Avalon, so as a community we can answer these questions: generallyintelligent.com/launch/ -
Kenneth Stanley
@kenneth0stanley
This argument is actually a domain-specific echo of Why Greatness Cannot Be Planned, where “AGI” is an ambitious objective and “what can you use this tech for” refers to nearby stepping stones.
François Chollet
@fchollet
The AGI discourse is counterproductive in many ways, one being that it forces you to counter-argue, "no, this avenue cannot lead to general intelligence because XYZ", whereas the much more interesting question is "what can you use this tech for?" -- not "will this lead to AGI?"
-
Kenneth Stanley reposted
Daniel
@public_daniel_
Me getting hyped up every time I watch @kenneth0stanley lecture (latest one was fantastic) youtu.be/qEZPqm1fQxI
-
Kenneth Stanley reposted
Julian Togelius
@togelius
Wow! Stable Diffusion PicBreeder when?
Simon Colton
@SimonGColton
A thread of 44 weird and wonderful images I produced recently with an evolutionary approach to #stablediffusion. Starting from scratch with no prompt ever seen, evolving using mutation. Notebook soon.
#aiart #stableevolusion #computationalcreativity @StableDiffusion
More 👇👇 -
Kenneth Stanley
@kenneth0stanley
Art in the Sciences of the Artificial: Is AI not only a science but also a branch of art? I spoke for the first time on this topic (in person!) to the always-stimulating @TTIVanguard conference, which also made an excellently-produced recording: www.youtube.com/watch?v=qEZPqm1fQxI
-
Kenneth Stanley
@kenneth0stanley
It was my privilege to be interviewed by @shaneparrish for the fantastic @TKPPodcast . Shane is a very talented interviewer and led the conversation in many fascinating directions. Check it out here: fs.blog/knowledge-project-podcast/kenneth-stanley/
-
Kenneth Stanley
@kenneth0stanley
I spent 2 hours of in-depth conversation with the @experilearning community about implications of Why Greatness Cannot Be Planned for *education*. First time to have such a great opportunity to go deeply on this important topic: www.youtube.com/watch?v=K4rKlPQd13A Thanks @experilearning!
-
Kenneth Stanley reposted
hardmaru
@hardmaru
Some personal news: After six years at Google, I decided it was time for me to leave and try something new again.
I had a fantastic time at Google Brain, and I’ll miss my friends, collaborators, and hanging out at the microkitchens! -
Kenneth Stanley reposted
Joel Lehman
@joelbot3000
Something I've believed for awhile but difficult to articulate: Open-endedness may be powerful microcosm for studying/navigating tension between unbridled creativity (e.g. blue-sky science) and dangers of catastrophic outcomes (existential risk)
e.g. see: arxiv.org/abs/2006.07495
Kenneth Stanley
@kenneth0stanley
That is why I think open-endedness research is in effect AI safety research: because a deeply open-ended process is inevitable when humans & AIs intersect in the real world, understanding what it means to constrain processes that are fundamentally unchained is the key question.
-
Eric Jang
@ericjang11
the wellspring of creativity and optimization that has sprung from the open-sourcing of large models reminds me of @kenneth0stanley 's "why greatness cannot be planned". GANbreeder and people hacking on SD create many things people couldn't even imagine, much less search for
Post not available here, view on X
Kenneth Stanley
@kenneth0stanley
That is why I think open-endedness research is in effect AI safety research: because a deeply open-ended process is inevitable when humans & AIs intersect in the real world, understanding what it means to constrain processes that are fundamentally unchained is the key question.
-
Kenneth Stanley reposted
Eric Jang
@ericjang11
the wellspring of creativity and optimization that has sprung from the open-sourcing of large models reminds me of @kenneth0stanley 's "why greatness cannot be planned". GANbreeder and people hacking on SD create many things people couldn't even imagine, much less search for
-
Kenneth Stanley
@kenneth0stanley
I’m really looking forward to this discussion next week as I haven’t had much opportunity so far to discuss exclusively implications for education & learning of Why Greatness Cannot Be Planned, and I think there are a lot. Thanks to @experilearning & @MoritzW42 for organizing!
Jamesb
@experilearning
Will be moderating (w/ @MoritzW42) a zoom discussion with @kenneth0stanley on The Myth of the Objective, learning and education!
Time: Oct 1st 12:00 PM Pacific Time
zoom.us/j/97761542641?pwd=TzJiZUF2RHcyQ0hSeUR6TUtTbkoyUT09
ID: 977 6154 2641
Passcode: 670624
My summary of Ken's book: www.youtube.com/watch?v=9Uc2Rvhn1wc -
Kenneth Stanley
@kenneth0stanley
I had a far-reaching conversation on the objective paradox on @superdatasci with @JonKrohnLearns , who put a lot of thought and preparation into the show! Here's the link: www.superdatascience.com/611
-
Kenneth Stanley reposted
Tim Rocktäschel
@_rockt
We are hiring for @DeepMind’s Open-Endedness team. If you have expertise in topics such as RL, evolutionary computation, PCG, quality diversity, novelty search, generative modelling, world models, intrinsic motivation etc., then please consider applying!
boards.greenhouse.io/deepmind/jobs/2999687 -
Kenneth Stanley reposted
MeleeNeat
@MeleeNeat
Working on a new stream feature to allow viewers to 'claim' interesting models they find on stream. Once that is implemented, I'll be introducing ways for viewers to interact with their collection.
-
Kenneth Stanley
@kenneth0stanley
This was a really fun talk and discussion at @UCL_DARK - very honored to speak! And the topics I covered are ones I've rarely spoken about previously (including new work), so it's quite unique. Thanks to the whole lab and especially @LauraRuis for the organizing!
UCL DARK
@UCL_DARK
We extend our deepest gratitude to Kenneth Stanley (@kenneth0stanley) for his amazing and thought-provoking talk on “Novel Opportunities in Open-Endedness” during the @UCL_DARK Invited Speaker Series.
Now available on our YouTube channel: youtu.be/T08wc4xD3KA -
Kenneth Stanley
@kenneth0stanley
Little-discussed challenge for Large Language Models: No clear path to learn the concept of novelty from data alone - novelty is a function of global chronology (not captured in dataset) and *all* that came before. Note the novelty instinct is the foundation of human creativity.
-
Kenneth Stanley reposted
rohit
@krishnanrohit
🚨New essay🚨!
Nothing worth discovering was discovered without a sense of play dominating its creation, as @kenneth0stanley said in his book.
And thus on the purpose of life; being curious, to play, and rekindling the explorer spirit!
www.strangeloopcanon.com/p/embrace-the-curious-life -
Kenneth Stanley reposted
Thomas Miconi
@ThomasMiconi
Two recent papers, one at @icmlconf 2022, the other at @NeuroCellPress, independently hit on the same modification of differentiable plasticity / fast weights:
Learn not just how to learn, but also how to forget! -
Kenneth Stanley
@kenneth0stanley
I just had a very interesting and wide-ranging conversation on Why Greatness Cannot Be Planned with the @LTCWRK community, hosted thoughtfully by @BlasMoros . Most of it is forward focused on what comes next after we accept the book's perspective. www.youtube.com/watch?v=z8x5VYF-pIw&t=471s&ab_channel=The…
-
Kenneth Stanley reposted
Eleni Nisioti
@nisioti_eleni
Excited to be at @GeccoConf next week to ask and answer: how far can a simple evolutionary model go in capturing the interplay of environmental dynamics and adaptation?
Work done with @Clement_MF_ at @FlowersINRIA
Paper arxiv.org/abs/2202.08834 and code github.com/eleninisioti/ClimateAndLearning -
Kenneth Stanley reposted
hardmaru
@hardmaru
I believe Text2Image models like #dallemini has a Novelty Search element that’s similar to PicBreeder
Unlike Picbreeder, the “Gene” is the natural language embedding space.
Talk youtu.be/dXQPL9GooyI?t=131
Slides wiki.santafe.edu/images/3/34/Stanley_innovation_workshop1…
More discussion here:
Kenneth Stanley
@kenneth0stanley
Agreed, the connection is unmistakeable! Even more, in some sense all interesting search spaces are like Picbreeder once you start seeing some of their structure unfold.
-
Kenneth Stanley reposted
Blas
@BlasMoros
One of the more impactful books I’ve read recently is @kenneth0stanley's Why Greatness Cannot Be Planned
He argues that objectives can be counterproductive for ambitious and innovative ideas
It refined when I implement objectives and when I simply explore
Key takeaways below: -
Post not available here, view on XPost not available here, view on X
Kenneth Stanley
@kenneth0stanley
Agreed, the connection is unmistakeable! Even more, in some sense all interesting search spaces are like Picbreeder once you start seeing some of their structure unfold.
-
Kenneth Stanley reposted
Anna Johnson
@annajohnson
Oh wow! Just got a huge a-ha moment when @kenneth0stanley explained the Picbreeder experiment.
-
Kenneth Stanley reposted
Invest Like the Best
@InvestLikeBest
"There is no good strategy to achieve a specific thing that we don't know how to do...
You have to understand that a successful inventor is somebody who's open-minded & willing to keep a repertoire of stepping stones around without knowing which one is going to be the payoff." -
Kenneth Stanley reposted
hardmaru
@hardmaru
The most interesting and viral images you see produced by text-to-image models are not merely the results of the deep learning models themselves, but rather the result of a complex feedback loop between a human neural net🧠 interacting with an artificial neural net🤖.
🧵Thread👇 -
Kenneth Stanley reposted
Tom Morgan
@tomowenmorgan
How many podcast episodes have you listened to that could change the way you live your whole life?
Today's @InvestLikeBest with @patrick_oshag and @kenneth0stanley could be one.
👇 -
Kenneth Stanley reposted
austin
@austin_hurwitz
Great listen from @kenneth0stanley & @patrick_oshag
Objective orientation & consensus destroy innovation by focusing on the most logical path
Innovation is almost always an unintended outcome as the result of previously disconnected stepping stones
open.spotify.com/episode/34gfbyX5h6OoCk1AhRa7XK -
Kenneth Stanley
@kenneth0stanley
Original and thoughtful questions, broad-ranging discussion - a real privilege to appear on @InvestLikeBest with @patrick_oshag !
Patrick OShaughnessy
@patrick_oshag
My conversation with @kenneth0stanley about why you shouldn’t pursue big hairy audacious goals
I hope this stirs up some controversy b/c the implications of the concept are large
That he “discovered” this idea via AI research fascinates me
Enjoy!
www.joincolossus.com/episodes/88083127/stanley-greatness-… -
Kenneth Stanley reposted
Bert Chan
@BertChakovsky
A new type of Lenia
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Introducing Video PreTraining (VPT): it learns complex behaviors by watching (pretraining on) vast amounts of online videos. On Minecraft, it produces the first AI capable of crafting diamond tools, which takes humans over 20 minutes (24,000 actions) 🧵👇openai.com/blog/vpt
This media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
New paper from our team @OpenAI: “Evolution through Large Models.” Idea: Large language models trained on code can propose mutations to programs of unprecedented coherence. Broad implications for EC,Genetic Programming,RL,deep learning,open-endedness. arxiv.org/abs/2206.08896 1/4
Post not available here, view on X
Kenneth Stanley
@kenneth0stanley
When mutation becomes more like an intelligent human, previously inaccessible spaces become accessible. New data can be generated, and even used to train new deep models in a new domain, e.g. 2D ambulating robots in this work. (It even reinvented the wheel!) 3/4
Kenneth Stanley
@kenneth0stanley
Worth highlighting fundamental implications for Genetic Programming (GP) - modern languages like Python become seamlessly and efficiently evolvable without any special provision. Broad new applications likely opened up! @moorejh @leespector @moshesipper @UnaMayMIT @WBanzhaf 4/4
-
Kenneth Stanley
@kenneth0stanley
Though comforting, following a path illuminated by a set of clear metrics is risky because other people can follow those metrics just as easily. Through its defiance of easy measure, the path of interestingness is the path of the brave, and most often walked alone.
-
Kenneth Stanley reposted
Pierre-Yves Oudeyer
@pyoudeyer
This unifies diverse approaches ranging from goal-conditioned Deep RL to Quality-Diversity algorithms @jb_mouret @CULLYAntoine @jeffclune @kenneth0stanley
-
Kenneth Stanley reposted
hardmaru
@hardmaru
In a new blog post by @yujin_tang @kazunori_279 @alanyttian, they demonstrate how EvoJAX, a new hardware-accelerated neuroevolution tool, can help solve difficult combinatorial optimization problems that arise in supply chain logistics at scale with TPUs:
cloud.google.com/blog/topics/developers-practitioners/evo…
hardmaru
@hardmaru
We just released EvoJAX, a hardware-accelerated neuroevolution toolkit built on top of JAX!
EvoJAX can run a wide range of evolution experiments within minutes on a TPU/GPU, compared to hours or days on CPU clusters.
Check it out:
github.com/google/evojax
arxiv.org/abs/2202.05008 -
Kenneth Stanley reposted
Danijar Hafner
@danijarh
Excited to share Director, a practical, general, and interpretable reinforcement learning algorithm for learning hierarchical behaviors from pixels!
Director explores and solves long-horizon tasks with very sparse rewards by breaking them down into internal subgoals.
Thread 👇 -
Kenneth Stanley
@kenneth0stanley
This interesting statement is more hypothesis than conclusion.Both evolution&GD create high-d structs, but the proposed analogy hinges on their internal organization. Because how X is discovered affects its representation, we can't assume big=similar.Merits further research! 1/2
Chris Olah
@ch402
The elegance of ML is the elegance of biology, not the elegance of math or physics.
Simple gradient descent creates mind-boggling structure and behavior, just as evolution creates the awe inspiring complexity of nature.
Kenneth Stanley
@kenneth0stanley
For an example of how the way something is found can dramatically impact how it is ultimately represented, see eplex.cs.ucf.edu/papers/woolley_gecco11.pdf (On the Deleterious Effects of A Priori Objectives on Evolution and Representation). 2/2
Kenneth Stanley
@kenneth0stanley
By the way, that doesn't necessarily mean GD can't yield biological-like representations. But it might mean the way the search is formalized might matter a lot, and conventional formulations might not do it.
-
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Many don't know that Hypernetworks, NeRF, SIREN, etc. approaches all originate from a method first popularized in the evolutionary computation community: Compositional Pattern Producing Networks by @kenneth0stanley. If you haven't already, read: eplex.cs.ucf.edu/papers/stanley_gpem07.pdf
-
Kenneth Stanley
@kenneth0stanley
Interesting to see a new RL-focused QD approach!
Tom Zahavy
@TZahavy
Excited to share DOMiNO, a method to discover qualitative-diverse policies using a single latent-conditioned architecture and the "reward is enough" principle.
Read more about it here: arxiv.org/pdf/2205.13521.pdf
DOMiNO's🍕 in Walker walk: -
Kenneth Stanley
@kenneth0stanley
This point from @hardmaru is why understanding open-endedness will ultimately prove fundamental to establishing a healthy dynamic as creative proliferation is increasingly dominated by machines.
hardmaru
@hardmaru
But unlike previous trends, machine learning models are constantly updated with new data, which is produced by our collective intelligence reflecting the current state of our culture. If most of this new creative content is made using ML, it will lead to this weird feedback loop.
-
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Most ANNs fail catastrophically if inputs are presented in an ordering unseen during optimization. In our recent @GeccoConf paper, @JoachimWinther and myself study what are the minimal network models to create input/output permutation-invariant agents.
arxiv.org/abs/2205.07868 -
Kenneth Stanley reposted
Samuel Arbesman
@arbesman
Using a bit of inspiration from @kenneth0stanley and @joelbot3000 on 'Why Greatness Cannot Be Planned' (and @gordonbrander), I argue that we need these outlier roles to help with less directed exploration of idea spaces:
arbesman.substack.com/p/-outlier-roles-in-venture-capital -
Kenneth Stanley
@kenneth0stanley
Some news: It has been a great privilege to contribute to the outstanding innovation at OpenAI and our team did exciting work (some out soon!), but I've decided the time is right for me to move on and to take this opportunity to really reflect on what I want to do next!
-
Kenneth Stanley reposted
Piero Molino
@w4nderlus7
I'm excited to announce Predibase, the enterprise declarative machine learning platfom. Bulding on top of the open source foudnations of Ludwig and Horovod to bring machine learning and data closer together. medium.com/predibase/introducing-predibase-the-enterprise… #machinelearning #datascience #deeplearning
-
Kenneth Stanley reposted
Patrick OShaughnessy
@patrick_oshag
The concept of evolutionary computing and specifically “the never ending algorithm” that @kenneth0stanley discussed here starting at 13:02 is an incredible idea
Reminds me of “inventing on principle”. What would be an “algorithm” you’d run forever ?
youtu.be/o5c4CzHyKns -
Kenneth Stanley
@kenneth0stanley
Neuroevolution is still a stepping stone to many important insights
MeleeNeat
@MeleeNeat
Been quite on here, but still making progress on mario and smash brothers w/ HyperNeat + NoveltySeach. Nearly completing first two levels in 1 life. ~4 days of training.
-
Kenneth Stanley reposted
Germán Kruszewski
@germank
Simulating a process that is analogous to the creation of all of nature is called open-ended evolution. If you haven't heard about it before, you should probably check out @kenneth0stanley's talk at @aloeworkshop in #iclr2022. Then, come by the @cells2societies workshop, where...
-
Kenneth Stanley
@kenneth0stanley
Excited to join the Workshop on Agent Learning in Open-Endedness (@aloeworkshop) at ICLR tomorrow! I'll answer questions for my invited talk during the allocated slot at 10:45am PT (pre-recorded talk & chat links at iclr.cc/virtual/2022/workshop/4547). Then in live zoom panel at 11:15am!
-
Kenneth Stanley reposted
Joseph Suarez 🐡
@jsuarez
Interested in expressive, open-ended RL environments?
Swing by the Neural MMO poster (3:20 pm ET) @aloeworkshop to hear about the latest v1.6 release featuring profession and exchange systems that enable agents to specialize! -
Kenneth Stanley
@kenneth0stanley
Thank you @savvyRL , this was a fun talk to put together and your thoughtful words mean a lot!
Rosanne Liu
@savvyRL
I'm getting ahead of myself, but this #iclr2022 talk by @kenneth0stanley "Why Open-Endedness Matters to Machine Learning" is like poetry 🥲 Beautifully delivered & full of amazing #dalle2 illustrations
To appear on Friday @aloeworkshop (already online):
iclr.cc/virtual/2022/workshop/4547 -
Kenneth Stanley reposted
ALOE Workshop
@aloeworkshop
With ALOE just 3 days away, we shine the spotlight on @joelbot3000, research scientist @OpenAI + longstanding pioneer of open-endedness research in ML. His research has shaped many paths toward open-endedness, including novelty search and neuroevolution.
-
Kenneth Stanley reposted
Tim Rocktäschel
@_rockt
Make sure to come to the @aloeworkshop at @iclr_conf 2022 this Friday and check out the full list of inspiring work on open-ended learning that will be presented👇
ALOE Workshop
@aloeworkshop
🌿We are excited to share with you all of the wonderful ideas in the submissions to the ALOE workshop this year. You can find the full list of accepted papers here:
sites.google.com/view/aloe2022/accepted-papers -
Kenneth Stanley reposted
ALOE Workshop
@aloeworkshop
As the ALOE workshop approaches (Fri, April 29), we spotlight our invited speaker @risi1979, professor at ITU Copenhagen & co-founder of @modl_ai. His work has advanced many areas related to open-endedness: evolution, meta-learning, generalization, and self-organizing systems.
-
Kenneth Stanley reposted
Josh Bongard
@DoctorJosh
The first of our three @GeccoConf publications:
Evolving continuous cellular automata to yield stable motile patterns.
By lab member @RiveSunder.
arxiv.org/abs/2204.07541 -
Kenneth Stanley reposted
Aleksa Gordić (水平问题)
@gordic_aleksa
[🧠new video 🥳] NeuroEvolution of Augmenting Topologies (NEAT) and Compositional Pattern Producing Networks (CPPN) papers explained!
Inspired by evolution, genetics, and developmental biology!
YT: www.youtube.com/watch?v=vf18FLdKkY4&ab_channel=TheAIEpiphany
@kenneth0stanley -
Kenneth Stanley
@kenneth0stanley
Very cool use of QD (quality diversity) algorithms to find diverse functional Lenia creatures!
Morgan
@morgangiraud
And even more importantly use the awesome line of work on Quality-diversity by @jb_mouret @CULLYAntoine @SDoncieux @kenneth0stanley and @jeffclune to find a huge amount of creatures!
-
Kenneth Stanley reposted
OpenAI
@OpenAI
Our newest system DALL·E 2 can create realistic images and art from a description in natural language. See it here: openai.com/dall-e-2/
-
Kenneth Stanley
@kenneth0stanley
A new movement?
Patrick OShaughnessy
@patrick_oshag
The anti-goal starter kit:
🎥 Inventing on Principle m.youtube.com/watch?v=PUv66718DII
📕 Why Greatness Cannot Be Planned www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/…
✍️ Growth Without Goals www.joincolossus.com/blog/posts/growth-without-goals -
Kenneth Stanley
@kenneth0stanley
I did it again - second time on @MLStreetTalk ! It was so much fun the first time that I couldn’t resist. Just as stimulating the second time around, with entirely new topics covered. Thanks to @ecsquendor and @DoctorDuggar !
Machine Learning Street Talk
@MLStreetTalk
This is a special day! We speak with @kenneth0stanley for the second time about subjectivity, art and open-endedness. Ken gave us a much deeper insight into his philosophy on the second time round. www.youtube.com/watch?v=DxBZORM9F-8
-
Kenneth Stanley reposted
Aleksa Gordić (水平问题)
@gordic_aleksa
[🧠new video 🥳] "Paired Open-Ended Trailblazer (POET)" paper explained - oldie but a goldie.
YT: youtu.be/1MnkDLHzzvQ
@ruiwang2uiuc @joelbot3000 @jeffclune @kenneth0stanley -
Kenneth Stanley reposted
Aaron Dharna
@_aadharna
In Open-Ended learning, we try to teach agents generalized skills. How important is it for agents to train and transfer between different types of environments?
We investigate this question in PINSKY -- a POET-like algorithm.
Paper link: arxiv.org/abs/2203.10941
1/6 -
Kenneth Stanley reposted
Minqi Jiang
@MinqiJiang
🧬 For ACCEL, we made an interactive paper to accompany the typical PDF we all know and love. "Figure 1" is a demo that lets you challenge our agents by designing your own environments! Now you can also view agents from many training runs simultaneously.
accelagent.github.io -
Kenneth Stanley
@kenneth0stanley
Interesting thoughts here are reminiscent of compositional pattern producing networks (CPPNs), which take advantage of the fact that conventions of symmetry and regularity naturally arise from function composition: eplex.cs.ucf.edu/papers/stanley_gpem07.pdf
Chico Camargo
@evoluchico
Have you ever noticed how nature seems to love symmetry?🔺🟥🔴
Evolution has literally trillions of shapes to pick from, and yet, biological structures often show symmetry and simplicity.
This is the story of the discovery that completely changed how I see biology. 🧵 -
Kenneth Stanley reposted
Olaf Witkowski
@okw
Eloquent post by @gmusser quoting @kenneth0stanley and yours truly on the promise of Open-Ended #ai and the secret of value of miscommunication 👉 fqxi.org/community/forum/topic/3751 fqxi.org/community/forum/topic/3751
-
Kenneth Stanley
@kenneth0stanley
This news is very sad. I was inspired by Julian’s pioneering evolution of French flags as I became fascinated with indirect encoding. We had many great conversations.
Artificial Life journal
@ARTL_journal
We are very saddened by the news of Julian Miller's death. He was a key innnovator in Natural Computing and inventor of Cartesian Genetic Programming. He will be missed by the Artificial Life community, as well as by his friends and colleagues at York University @YCCSA.
-
Kenneth Stanley
@kenneth0stanley
Inspiring work here for anyone interested in POET or open-endedness! Congrats to @jparkerholder and his coauthors! I recommend taking a look:
Jack Parker-Holder
@jparkerholder
Evolving Curricula with Regret-Based Environment Design
Website: accelagent.github.io
Paper: arxiv.org/abs/2203.01302
TL;DR: We introduce a new open-ended RL algorithm that produces complex levels and a robust agent that can solve them (e.g. below).
Highlights ⬇️! [1/N] -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Excited to announce the 2nd edition of our Minecraft Open-Endedness Challenge @GeccoConf! Can you create an algorithm that produces a never-ending stream of increasingly complex artefacts?
More info: evocraft.life
Deadline: June 1st
Below highlights from last year. -
Kenneth Stanley reposted
ALOE Workshop
@aloeworkshop
We are pleased to share that Lisa Soros (@err_more) will be giving a talk at ALOE 2022. Lisa’s work spans ML and artificial life, where her research has advanced fundamental understanding of ideas in open-endedness and quality-diversity.
-
Kenneth Stanley
@kenneth0stanley
On consciousness I notice many dismiss it for lacking objective measure. But that’s precisely its fascination: it is the phenomenon of subjectivity itself, literally what it’s like from the inside. If science lacks the tools it is an indictment of science, not of consciousness.
-
Kenneth Stanley reposted
hardmaru
@hardmaru
We just released EvoJAX, a hardware-accelerated neuroevolution toolkit built on top of JAX!
EvoJAX can run a wide range of evolution experiments within minutes on a TPU/GPU, compared to hours or days on CPU clusters.
Check it out:
github.com/google/evojax
arxiv.org/abs/2202.05008 -
Kenneth Stanley reposted
Bert Chan
@BertChakovsky
In this fun project, we trained artificial life using machine learning!
Lenia creatures learn to survive obstacles using gradient descent (Neural CA) & curriculum learning (IMGEP), generalize well to unseen tasks.
Full video + blog post + interactive demo: developmentalsystems.org/sensorimotor-lenia/ -
Kenneth Stanley reposted
ALOE Workshop
@aloeworkshop
Today, we spotlight @togelius, prof @NYUGameLab and co-founder of @modlai. His research program applies AI to video game design—revealing key insights both in how AI can design and mediate human experience in simulated worlds + how such simulations can lead to more general AI.
-
Kenneth Stanley
@kenneth0stanley
Looking forward to joining a bunch of great speakers to talk about open-endedness at @aloeworkshop at ICLR 2022!
ALOE Workshop
@aloeworkshop
Kicking off our speaker spotlights is @kenneth0stanley, a pioneer of open-endedness in ML. He co-authored Why Greatness Cannot Be Planned w/ @joelbot3000 (also speaking). They argue open-ended search can lead to better solutions than direct optimization. link.springer.com/book/10.1007/978-3-319-15524-1
-
Kenneth Stanley reposted
Jagdeep Bhatia @ CORL 26
@JagdeepBhatia8
Exciting news: we've just released the codebase for Evolution Gym -- a benchmark for design and control co-optimization of voxel-based soft robots 🥳!
You can find docs, examples, and tutorials on our website!
code: github.com/EvolutionGym/evogym
docs: evolutiongym.github.io -
Kenneth Stanley reposted
Charles Ofria
@CharlesOfria
Congratulations to all of the newly elected @aaas fellows announced today! MSU has a record nine inductees and I'm proud to say that I made the cut. msutoday.msu.edu/news/2022/record-year-for-spartan-scient…
-
Kenneth Stanley reposted
OpenAI
@OpenAI
We're introducing embeddings, a new feature of our API that distills relationships between concepts, sentences, and even code in a simple numerical representation — for more powerful search, classification, and recommendations. openai.com/blog/introducing-text-and-code-embeddings/
-
Kenneth Stanley reposted
Thomas Miconi
@ThomasMiconi
It's out!
Learning to acquire novel cognitive tasks with evolution, plasticity and meta-meta-learning
arxiv.org/abs/2112.08588
TL/DR: We evolve plastic networks that can automatically acquire novel, simple cognitive tasks from rewards alone, like this:
Thomas Miconi
@ThomasMiconi
This is an evolved plastic RNN, learning a simple cognitive task (delayed match-to-sample) over successive trials.
No backprop or gradients, just the network's own evolved plasticity and connections, learning automatically.
The real point is: it never saw this task before. -
Kenneth Stanley reposted
Pierre-Yves Oudeyer
@pyoudeyer
To start the new year with fun, here's a gift from the @flowersInria team:
👉An interactive online app to test generalization capabilities of #deeprl agents! 🎁🤖
developmentalsystems.org/Interactive_DeepRL_Demo/
www.youtube.com/watch?v=lsfCnLajims -
Kenneth Stanley reposted
-
Kenneth Stanley reposted
Tim Rocktäschel
@_rockt
The place to be if you are working on Open-ended Learning. Can't wait to hear from @kenneth0stanley @natashajaques @togelius @risi1979 Lisa Soros @maxjaderberg @joelbot3000 @danijarh and @Smearle_RH about their exciting work and perspectives at our @iclr_conf 2022 workshop!
ALOE Workshop
@aloeworkshop
We have an incredible set of speakers lined up for ALOE 2022. Together they represent a wide-ranging vista of open-endedness research. Over the next few weeks, we will feature brief profiles about their work. Stay tuned!
-
Kenneth Stanley reposted
ALOE Workshop
@aloeworkshop
We have an incredible set of speakers lined up for ALOE 2022. Together they represent a wide-ranging vista of open-endedness research. Over the next few weeks, we will feature brief profiles about their work. Stay tuned!
-
Kenneth Stanley reposted
Santa Fe Institute
@sfiscience
🌄Frontiers in Evolutionary Computation: A Workshop Report
by SFI's Tyler Millhouse, @MelanieEMoses, @MelMitchell1
Computer scientists & biologists share insights into the nature of evolution, evolutionary computation, & frontiers for further research:
arxiv.org/abs/2110.10320 -
Kenneth Stanley
@kenneth0stanley
Sounds like Why Greatness Cannot Be Planned - nice initiative here!
Patrick Collison
@patrickc
On top of that, consensus-oriented approval processes mean that many scientists don’t even have the freedom to focus on their best ideas. So, first and foremost, Arc is about returning science to the scientists.
-
Kenneth Stanley reposted
OpenAI
@OpenAI
We trained a research version of GPT-3 that can search the web, synthesize information, and cite its sources to provide more accurate answers to questions. openai.com/blog/improving-factual-accuracy/
-
Kenneth Stanley reposted
ALOE Workshop
@aloeworkshop
Announcing the first Agent Learning in Open-Endedness (ALOE) Workshop at #iclr2022!
We're calling for papers across many fields: If you work on open-ended learning, consider submitting. Paper deadline is February 25, 2022, AoE. sites.google.com/view/aloe2022. -
Kenneth Stanley reposted
OpenAI
@OpenAI
Developers can now create a custom version of GPT-3 for their applications with a single command. Fine-tuning GPT-3 on your data improves performance for many use cases. See results👇 openai.com/blog/customized-gpt3/
-
Kenneth Stanley reposted
hardmaru
@hardmaru
Evolution Gym: A Large-Scale Benchmark for Evolving Soft Robots
Seen at #neurips2021: A benchmark for design and control co-optimization based on voxel-based soft robots! Written in C++ with a Python/Gym interface, and includes a set of 30 diverse tasks.
openreview.net/forum?id=lM2971LAwV -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Finally done with my first blog post "The Future of Artificial Intelligence is Self-Organizing and Self-Assembling"!
sebastianrisi.com/self_assembling_ai/
Covering work from our group and others on the combination of ideas from deep learning and self-organizing systems. -
Kenneth Stanley
@kenneth0stanley
What a gratifying tweet to read! Thank you @mcuban for the sentiment!
Mark Cuban
@mcuban
This TDS podcast with Ken Stanley discussing his new book didn't set out to be about how great entrepreneurs discover their route, but it was amazing and did so. www.amazon.com/dp/3319155237?tag=foodguides-shop-20
-
Kenneth Stanley reposted
Josh Bongard
@DoctorJosh
The new #xenobot paper is up on PNAS now: www.pnas.org/content/118/49/e2112672118
w/ @dougblackiston, @Kriegmerica and @drmichaellevin -
Kenneth Stanley reposted
jrieffel dot bsky dot social
@jrieffel
Delighted that our (@jb_mouret,@CULLYAntoine, @togelius, @SDoncieux, Stefanos Nikolaidis, @tehqin17) GECCO '22 Workshop on Benchmarks for Quality Diversity Algorithms has been accepted. See you all in Boston!
-
Kenneth Stanley
@kenneth0stanley
Great questions from @jeremiecharris on the @TDataScience podcast! Very much enjoyed the conversation - thanks for having me!
Towards Data Science
@TDataScience
Why is it so hard to build creative AI, and what are the risks around open-ended learning strategies?
@kenneth0stanley joins @jeremiecharris on the TDS Podcast this week to discuss these (and a few more...) questions. towardsdatascience.com/ai-without-objectives-48b9c8e7b988 -
Kenneth Stanley reposted
Antoine Cully
@CULLYAntoine
Hi All, I am recruiting new PhD students for my Lab at Imperial College London. Our goal is to make robots more resilient and versatile with ML. So, if you are interested in Robotics and Learning algorithms (Quality-Diversity, Evolutionary Robotics, DeepRL), consider applying!
-
Kenneth Stanley reposted
OpenAI
@OpenAI
We introduced new safeguards to our API that make it possible to remove the waitlist. Now developers can sign up and explore GPT-3 right away. openai.com/blog/api-no-waitlist/
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
My keynote at CORL is now online. "Improving Robot and Deep Reinforcement Learning via Quality Diversity, Open-Ended, and AI-Generating Algorithms." Thanks again to the organizers for the invitation!
youtu.be/zpUD9rf5YaQ?t=15069 -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Can we develop more robust image recognition systems? Maybe by combining ideas from deep learning and collective intelligence? Come join our group as a postdoc and let's find out!
candidate.hr-manager.net/ApplicationInit.aspx?cid=119&Pro…
Application Deadline: November 18th -
Kenneth Stanley reposted
Sam Altman
@sama
Prediction: the coming generation of AI-assisted creative tools are going to be surprisingly great.
This idea of “say what you want, and the computer creates it” will be extended from programming to art.
(Demo of the code version: www.youtube.com/watch?v=Zm9B-DvwOgw) -
Kenneth Stanley reposted
Karl Cobbe
@karlcobbe
Excited to share our recent work @OpenAI: training large language models (like GPT-3) to solve grade school math problems much more effectively! openai.com/blog/grade-school-math/
-
Kenneth Stanley reposted
Joseph Suarez 🐡
@jsuarez
The Neural MMO Platform for Massively Multiagent -- NeurIPS 2021 Dataset/Benchmark paper, now on arXiv: arxiv.org/abs/2110.07594
Summarizes architectural+experimental findings from the last ~4 years of development!
Video: www.youtube.com/watch?v=hYYA8_wFF7Q
More at neuralmmo.github.io -
Kenneth Stanley
@kenneth0stanley
HyperNEAT still teaming up with novelty search, nice!
MeleeNeat
@MeleeNeat
Was able to "solve" 1-1 in Super Mario Brothers with hyperNEAT & Novelty search in 70 generations - population size 500.
32x30 grayscale input (downscaled 256 x 240 input by 8)
5x5 hidden layer 1
9x9 hidden layer 2 w/ cyclic connections
1x12 output layer -
Kenneth Stanley reposted
Georgios N. Yannakakis
@yannakakis
Game bots that both play and "feel" like humans? Proudly presenting our recent @acii_conf paper that views affect modelling as an RL problem and applies Go-explore (by @kenneth0stanley @jeffclune et al.) to generate human-like play and arousal patterns: arxiv.org/pdf/2109.13388.pdf
-
Kenneth Stanley reposted
Tim Taylor
@drtimt
Yesterday's panel discussion on #openendedness hosted by @data_icmc is available to view on YouTube at youtu.be/hr_7SLG3GDA. Including some nice shout-outs to the #artificiallife community from @kenneth0stanley
-
Kenneth Stanley
@kenneth0stanley
Looking forward to joining the panel in 30 minutes!
-
Kenneth Stanley reposted
Matt Fontaine
@tehqin17
Excited to announce our work on differentiable quality diversity (DQD) has been accepted to #neurips2021 for an 𝗼𝗿𝗮𝗹 presentation!
Matt Fontaine
@tehqin17
Excited to share some recent work with @snikolaidis19 on differentiable quality diversity (DQD).
pdf: arxiv.org/pdf/2106.03894.pdf
abs: arxiv.org/abs/2106.03894
code: github.com/icaros-usc/dqd -
Kenneth Stanley reposted
UCL DARK
@UCL_DARK
Interested in Open-Ended Learning? Join the panel discussion on Open-Endedness featuring @UCL_DARK's @_rockt together with @maxjaderberg (@DeepMind), @kenneth0stanley (@OpenAI) and @danijarh (@UofT & @GoogleBrain).
🕰️ Oct 7th 11:00AM (-3:00 GMT)
🔗 open-endedness.netlify.app
Tim Rocktäschel
@_rockt
Really looking forward to discussing research on open-ended learning with @maxjaderberg, @danijarh and @kenneth0stanley. Thanks @_joaogui1 for organizing!
open-endedness.netlify.app -
Kenneth Stanley reposted
Maddie Simens
@maddiesimens
👋 Hey, you! Come work with me on product design at @OpenAI?
I joined OpenAI last year because the mission, opportunity, and team were so irresistibly exciting. We’re up to our ears in deeply interesting, high-impact design problems. -
Kenneth Stanley reposted
hardmaru
@hardmaru
The Eleventh International Conference on Artificial Intelligence in Music, Sound, Art and Design (EvoMUSART)
redd.it/pt4ehu -
Kenneth Stanley reposted
Cedric Rodriguez
@cedricrodrigz
Check out my interpretation of @kenneth0stanley's book 'Why Greatness Cannot Be Planned'. A great book where computer science concepts seem to be applicable to our daily lives. Let me know what you think! www.linkedin.com/pulse/myth-objective-cedric-rodriguez
-
Kenneth Stanley
@kenneth0stanley
This will be fun - a panel discussion on open-endedness on October 7th - looking forward to joining the conversation with some great panelists! Thanks @_joaogui1 for organizing!
-
Kenneth Stanley reposted
hardmaru
@hardmaru
“A drawing of a cat”
CLIP + ES + Triangles -
Kenneth Stanley
@kenneth0stanley
We're hiring a software engineer at OpenAI who will have the opportunity to work closely with our team, among others! More here: grnh.se/d0ea97c44us
-
Kenneth Stanley reposted
hardmaru
@hardmaru
I think the name NEAT is already taken :)
-
Kenneth Stanley reposted
hardmaru
@hardmaru
The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning
We explore RL agents that still work even when their observations get shuffled around a lot!
A fun paper w/ @yujin_tang
web attentionneuron.github.io
pdf arxiv.org/abs/2109.02869 -
Kenneth Stanley reposted
Aaron Dharna
@_aadharna
For the past 6 months, I've been building a library for open-ended coevolutionary(/multi-agent rl) style research (e.g. POET, PAIRED, arxiv.org/abs/2107.12808) on top of Ray/Rllib and @Bam4d's Griddly that doesn't require google-scale resources.
-
Kenneth Stanley
@kenneth0stanley
That means a lot to @joelbot3000 and I, @davempayne! Thank you!
dave payne
@davempayne
I’m totally taken by an obscure book on innovation called Why Greatness Cannot Be Planned. Feels like something big that will be in my head forever. Thanks so much for the recommendation @_cturner. a.co/4Z7VKdV
-
Kenneth Stanley reposted
OpenAI
@OpenAI
Today's live demo of Codex, our AI that translates natural language to code: www.youtube.com/watch?v=SGUCcjHTmGY
-
Kenneth Stanley reposted
Gautam Shroff
@gmshroff
Chap 11 of this excellent book by @kenneth0stanley should be required reading for all reviewers in AI conferences; and the whole book for anyone having to ‘manage’ research (like myself!). scholar.google.co.in/citations?view_op=view_citation&hl=e…
-
Kenneth Stanley reposted
Caitlin
@csgrasso10
Thank you again to our judges @jeffclune, Cecilia Laschi, @joelbot3000, @DrRituRaman, @sina_lana, @kenneth0stanley, and @JJVentrella
And to @cliffbohm for creating this stunning trophy! -
Kenneth Stanley reposted
Pierre-Yves Oudeyer
@pyoudeyer
Introducing TeachMyAgent, a Benchmark for Automatic Curriculum Learning in #deeprl @ #icml2021 !
PCG environments + #deeprl learners + diverse robot morphologies + many tests + 7 baselines
Web: developmentalsystems.org/TeachMyAgent/
Paper: arxiv.org/abs/2103.09815
Code: developmentalsystems.org/TeachMyAgent/ -
Kenneth Stanley reposted
Julian Togelius
@togelius
At @GeccoConf, Tae Jong Choi just presented his and mine paper on "Self-Referential Quality Diversity Through Differential Map-Elites"
As the name suggests, we describe a new algorithm that combines differential evolution and MAP-Elites
dl.acm.org/doi/pdf/10.1145/3449639.3459383 -
Kenneth Stanley
@kenneth0stanley
Great time talking to @jim_rutt about neuroevolution on the @JimRuttShow ! Thank you Jim!
-
Kenneth Stanley reposted
Joseph Suarez 🐡
@jsuarez
Announcing the Neural MMO Challenge, live...now! Create RL, scripted, or hybrid agents to complete high-level exploration, foraging, and combat tasks. We’ll drop them in a massively multiagent environment full of competing submissions.
www.aicrowd.com/challenges/the-neural-mmo-challenge -
Kenneth Stanley reposted
Georgios N. Yannakakis
@yannakakis
How about rebuilding MIT and Harvard campus areas to increase Bostoners' comfort? Introducing quality diversity for urban design via ARCH-Elites, with @TheodoreGalanos and @SentientDesigns. @GeccoConf @Prismarch_h2020
Video: www.youtube.com/watch?v=wLh69NLWsBo
Paper: arxiv.org/pdf/2104.08774 -
Kenneth Stanley
@kenneth0stanley
Very interesting how this paragraph from @JeffBezos does sound like something out of Why Greatness Cannot Be Planned. Good catch @MAghajohari !
Milad Aghajohari
@MAghajohari
This paragraph from a letter Jeff Bezos wrote to shareholders strongly reminds me of the whole idea behind following the gradient of interestingness that I learned about in the great podcast by @ecsquendor on @kenneth0stanley ideas.
-
Kenneth Stanley reposted
Christian Ulstrup
@curious_vii
I want a Twitter algo that surfaces interesting ideas, not just what people want to hear
-
Kenneth Stanley reposted
Sebastian Risi
@risi1979
A Picbreeder for Minecraft structures but grown with a neural cellular automata. Maybe interesting to @kenneth0stanley, @_joelsimon, @zzznah, @hardmaru, @BertChakovsky
-
Kenneth Stanley
@kenneth0stanley
Great time talking to @jim_rutt on his show about Why Greatness Cannot Be Planned! Thank you Jim for the broad and interesting discussion!
-
Kenneth Stanley reposted
Tim Scarfe
@ecsquendor
Small refresher on the concept of deception in objective gradients and @kenneth0stanley's work i.e. reward isn't enough IMO 😀 youtu.be/wNhaz81OOqw
-
Kenneth Stanley reposted
Caitlin
@csgrasso10
The Virtual Creatures Competition is back for 2021! And with an incredible panel of judges! Submit your virtual creatures by 𝗝𝘂𝗹𝘆 𝟮 to be reviewed by:
@jeffclune
Cecilia Laschi
@joelbot3000
@DrRituRaman
@sina_lana
@kenneth0stanley
@JJVentrella
virtualcreatures.github.ioThis media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
POET-inspired research crushing it on a real robot! Very impressive! It's great to see this work taken for a spin in the real world. PDF: www.research-collection.ethz.ch/bitstream/handle/20.500.1… POET: eng.uber.com/poet-open-ended-deep-learning/ www.youtube.com/watch?v=9j2a1oAHDL8 Congrats to the authors. Great work!
-
Kenneth Stanley
@kenneth0stanley
There will always be an inclination towards tightening up the rigor patrol to protect us from the insignificant masquerading as significant. But that obfuscates that rigor alone cannot decisively settle what is actually interesting. The future of AI depends more on THAT question.
Jacob Buckman
@jacobmbuckman
New blog post, "Please Commit More Blatant Academic Fraud": jacobbuckman.com/2021-05-29-please-commit-more-blatant-ac…
Yes, I'm serious. Blatant academic fraud might be our best shot at developing the future of artificial intelligence. -
Kenneth Stanley
@kenneth0stanley
Interesting thought on supervised vs. unsupervised. It also highlights another issue: The term "open-ended" deserves its own place alongside these venerable mainstays in the dialogue of the field.
François Chollet
@fchollet
A more interesting theoretical distinction between "supervised" and "unsupervised" learning could be:
1. Forms of learning driven by an explicit, known objective
vs.
2. Forms of learning driven by open-ended discovery and ever-shifting objectives elected by the system itself -
Kenneth Stanley reposted
MIT Technology Review
@techreview
An AI researcher at @Uber, Wang likes to leave the Paired Open-Ended Trailblazer, a piece of software he helped develop, running on his laptop overnight. eng.uber.com/poet-open-ended-deep-learning/
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Great article about AI-generating Algorithms ("getting AI to make itself"), including POET, GTNs, & ANML Also includes work & quotes from @ruiwang2uiuc (POET), Esteban Real (AutoML), & @janexwang (LRL), plus great work by @OpenAI Thanks for the fun interview & article @strwbilly!
MIT Technology Review
@techreview
Artificial intelligence might not need us to become smarter than we are. This thread is about how AI is learning how to create itself. www.technologyreview.com/2021/05/27/1025453/artificial-in…
-
Kenneth Stanley reposted
Greg Brockman
@gdb
Announcing a $100M fund for startups building on OpenAI API.
We’re particular excited about investing in application areas where AI can have a transformative impact, like healthcare, education, or climate change: openai.com/fund/ -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Our work Enhanced POET is an invited poster at this (excellent!) workshop in case you want to come ask questions or ask about any of our work. Thanks to the organizers for putting together such a wonderful event! eng.uber.com/enhanced-poet-machine-learning
Rose
@rose_e_wang
It's finally time! Join us tomorrow (Friday) for a series of exciting discussions on the roadmap to never-ending RL with the amazing @danijarh @aharutyu @joelbot3000 @pyoudeyer @NataliaDiazRodr @AleksandraFaust @hyogweon and more! 🥳🥳🥳
Schedule: sites.google.com/view/neverendingrl/home -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Excited to give a keynote at the ICLR Workshop on Neural Architecture Search tomorrow (May 7th). I will do a deep dive into Generative Teaching Networks and Synthetic Petri Dish. AMA in the Q&A. AI-GAs++! sites.google.com/view/nas2021 Work led by @felipesuch (GTNs) & @AdityaRawaI (SPD)
-
Kenneth Stanley reposted
Georgios N. Yannakakis
@yannakakis
Only a few days left (early-bird) to join @togelius and me on what promises to be a great journey through the various uses of AI in Games. Grab your seat and enjoy our unique panel of invited speakers, the diverse topics covered and the various activities hosted by @modl_ai.
AI and Games School 2026
@GameAISchool
Only 4 days left of our Early Bird deal! Grab your ticket today at school.gameaibook.org
We are going ONLINE this year with a great panel of industry experts from @unity3d @DeepMind @CAGames @King_Games @MSFTResearch @UbiMassive @AiDungeon SIE R&D (@Sony) @modl_ai and more! -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
We introduce Evolve&Merge, a genomic bottleneck algorithm that iteratively reduces the no. of Hebbian rules in a network, learning which connections should share the same rules: 384 rules can self-organize random weights in a few steps to control a quadruped! w/ @JoachimWinther
-
Kenneth Stanley
@kenneth0stanley
Thank you @NetThoughts for the opportunity to contribute my two cents on current grand challenges for AI to your article in @RealAAAI! It won't surprise many that I chose open-endedness.
Ganesh Mani
@NetThoughts
Great collaborating on this #ai #imagination. Of the people, by the people with machines, for the
people! @withfries2 @a16z Steve Cross @GeorgiaTech Tom Kalil Schmidt Futures @PROFVG @ISPPitt @frossi_t @ibm @kenneth0stanley @OpenAI #health #wealth #wisdom #ethics @NewYorker -
Kenneth Stanley reposted
ALIFE Conference 2027
@ALifeConf
#alife2021 Keynote Speaker Announcement #2
David Ha @hardmaru from Google Brain Tokyo.
"Artificial Life at Scale? Lessons from Machine Learning"
👉 robot100.cz/keynotes
#alife #machinelearning -
Kenneth Stanley reposted
Ádám Katona
@AdamKat0na
Check out our poster at @EvostarConf about why it is difficult to train indirect encoding with greedy learning algorithms.
We showed that Hypernetworks perform worse than direct encoding with greedy learning, but perform better with meta-learning. -
Kenneth Stanley reposted
Kai Arulkumaran
@kaixhin
Hiring an evolutionary computation researcher at Araya. Exciting collaboration with @GoodAIdev and @err_more (@crosslabstokyo) to explore evolutionary and open-ended algorithms to video game asset design! 🧬🕹️
research.araya.org/careers/ -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
So far confirmed sponsors for our Minecraft Open-Endedness Challenge are @modl_ai and @OpenAI!
You already know open-endedness is cool 😎 but if you need additional motivation to submit, we already have over $2000 USD in monetary awards 💸
More info: evocraft.life -
Kenneth Stanley reposted
Nemanja Rakicevic
@nrkcv
Excited to announce that our paper titled:
"Policy Manifold Search: Exploring the Manifold Hypothesis for Diversity-based Neuroevolution"
got a accepted to @GeccoConf #gecco2021!
🦎📜🎉
co-authored by @CULLYAntoine and @Petar_Kormushev -
Kenneth Stanley reposted
Maxime Allard
@allardmaxime079
I wrote a small introduction to Quality-Diversity Optimisation for anybody who wants to learn more about this pretty cool family of algorithms🤗 (of course I am biased😁). Hopefully this will spark more curiosity around this topic and its applications! maximeallard.lu/2021/03/24/quality-diversity-algorithms/
-
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Excited to share our work on Morphogenesis in Minecraft! We show that neural cellular automata can learn to grow not only complex 3D artifacts with over 3,000 blocks but also functional Minecraft machines that can regenerate when cut in half 🐛🔪=🐛🐛
PDF:arxiv.org/abs/2103.08737This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Adrien Ecoffet
@AdrienLE
“First return, then explore”, has been published in Nature! Working on Go-Explore with my co-first author @Joost_Huizinga as well as @joelbot3000, @kenneth0stanley and @jeffclune has been an amazing adventure and I am thrilled that we got to this point! www.nature.com/articles/s41586-020-03157-9
-
Kenneth Stanley reposted
Joost Huizinga
@Joost_Huizinga
Honored to say that our paper “First return, then explore”, has been published in the journal nature. Massive thanks to my co-authors @joelbot3000, @kenneth0stanley, @jeffclune and especially to my fellow first-author @AdrienLE! www.nature.com/articles/s41586-020-03157-9
-
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Excited to announce the first competition on open-endedness! w/ @DjordjeGrbic2, @rasmusbergpalm, @claire__aoi, @enasmel @ @GeccoConf
More info: evocraft.life
With a stellar panel of judges: @hardmaru, @err_more, @sina_lana, @kenneth0stanley, and @joelbot3000 🦾
Sebastian Risi
@risi1979
EvoCraft: A New Challenge for Open-Endedness!
Can we automatically search for increasingly complex artifacts/machines in an open-ended fashion in Minecraft?
PDF: arxiv.org/abs/2012.04751
Python API: github.com/real-itu/Evocraft-py
First competition on open-endedness coming: @GeccoConf!👇 -
Kenneth Stanley
@kenneth0stanley
I was just reading a book to my 6-year-old that briefly discussed Einstein. His incredulous response: "Einstein is real!?" That made me realize that the true mark of success is when your achievements reach such a level that people assume you must be a fictional character.
-
Kenneth Stanley reposted
Joseph Suarez 🐡
@jsuarez
Neural MMO v1.5: Large scale update, live now!
1000 -> 10,000 Time Horizons
80x80 ->1024x1024 Maps
128 -> 1024 Agents
Documentation: jsuarez5341.github.io
Discord: discord.gg/BkMmFUC
Github: github.com/jsuarez5341/neural-mmo
Demo: www.youtube.com/watch?v=d1mj8yzjr-w -
Kenneth Stanley reposted
Workshop on Open-Ended Evolution
@wsOEE
The Fourth Workshop on Open-Ended Evolution will take place at #alife2021 in July! It will primarily be an online event. More details and a call for papers will appear here soon...
-
Kenneth Stanley reposted
Matt Fontaine
@tehqin17
Attending #aaai2021? Come chat with me about how QD algorithms can help you explore latent space and find the content you want!
Sat. 2/6 8:45-10:30, 4:45-6:30 PST.
Room C Poster CE3
Paper: arxiv.org/abs/2007.05674 -
Kenneth Stanley reposted
Matt Fontaine
@tehqin17
Excited to release pyribs today, our new quality diversity (QD) library!
Website: pyribs.org
Documentation: docs.pyribs.org
GitHub: github.com/icaros-usc/pyribs
Fellow Contributors: @btjanaka @LunJohnZhang @snikolaidis19 -
Kenneth Stanley reposted
Rosanne Liu
@savvyRL
This Friday (since everyone is busy with ICML and I'm pathetically free) I am giving a talk at *my own* reading group -- extreme nepotism or what? 😬
Title: "Unconventional ways of training neural networks and what they teach us about model capacity"
mlcollective.org/dlct -
Kenneth Stanley reposted
Steve Coast
@SteveC
Awesome book to rethink your life by @kenneth0stanley
www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/… -
Kenneth Stanley
@kenneth0stanley
Appearing on this show was a great experience, exhilarating discussion, and also at times thought-provoking debate - thank you @MLStreetTalk! (And it leads with a very well-produced intro to our work going back a long ways. Much appreciation to @ecsquendor and all involved!)
Machine Learning Street Talk
@MLStreetTalk
Epic special edition. @kenneth0stanley on why greatness cannot be planned, abandoning objectives and open-endedness. @joelbot3000 @jeffclune @ykilcher
www.youtube.com/watch?v=lhYGXYeMq_E -
Kenneth Stanley
@kenneth0stanley
I did not contribute, but this NEAT-Gym package from Simon Levy and Coletta Fuller (neither on Twitter) may be interesting to some. It lets you run Python-based NEAT/HyperNEAT/ES-HyperNEAT/novelty search in OpenAI Gym: github.com/simondlevy/NEAT-Gym
-
Kenneth Stanley
@kenneth0stanley
Thanks for highlighting this paper Carlos! We were excited that such a simple principle could so effectively maintain open-ended diversity generation.
Carlos E. Perez
@IntuitMachine
Kennet Stanley's @kenneth0stanley paper that describes how diversity is generated via resource constraints is one of the more fascinating discoveries in 2020. dl.acm.org/doi/10.1145/3377930.3389809
-
Kenneth Stanley reposted
OpenAI
@OpenAI
We’ve developed two neural networks which have learned by associating text and images. CLIP maps images into categories described in text, and DALL-E creates new images, like this, from text.
A step toward systems with deeper understanding of the world. openai.com/multimodal/ -
Kenneth Stanley reposted
Ilya Sutskever
@ilyasut
Synthetic capybaras in different styles
openai.com/blog/dall-e/ -
Kenneth Stanley reposted
Julian Togelius
@togelius
To put it another way, if we know enough about what we’re trying to do to be able to plan it in advance, we’re not doing anything ambitious. In the words of @kenneth0stanley and @joelbot3000, “greatness cannot be planned”.
-
Kenneth Stanley reposted
Klas Pettersen 🤖🧠🤼
@kl4sp
My next proposal will be a one-liner:
"Greatness cannot be planned" 🤓 -
Kenneth Stanley reposted
MONTREAL.AI
@Montreal_AI
ONE Day! #aidebate2
Speakers: @rcalo, @YejinChoinka, @mmitchell_ai, @frossi_t, @drfeifei, @AmiiThinks, @luislamb, @yudapearl, @osazuwa, @kenneth0stanley, @celestekidd, @doristsao, @AdamMarblestone
Moderator: @GaryMarcus
RSVP: aidebate.eventbrite.ca
#montrealai #aidebate -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
EvoCraft: A New Challenge for Open-Endedness!
Can we automatically search for increasingly complex artifacts/machines in an open-ended fashion in Minecraft?
PDF: arxiv.org/abs/2012.04751
Python API: github.com/real-itu/Evocraft-py
First competition on open-endedness coming: @GeccoConf!👇 -
Kenneth Stanley reposted
Aditya Rawal
@AdityaRawaI
Looking forward to presenting our paper 'Synthetic Petri Dish: Rapid Surrogate to speed-up NAS' at the #neurips2020 Meta-Learning workshop tomorrow. w/ @joelbot3000, @felipesuch, @jeffclune and @kenneth0stanley
meta-learn.github.io/2020/ -
Kenneth Stanley reposted
Adam Marblestone
@AdamMarblestone
Why MOST research should NOT be objective-driven. To get to any one objective, you have to build on a library of stepping-stones that were discovered based on subjective interestingness unrelated to that or maybe any objective.
www.youtube.com/watch?v=dKazBM3b74I
Tapa Ghosh
@semiDL
If you read no other book in your life, this is the one
-
Kenneth Stanley reposted
Tapa Ghosh
@semiDL
If you read no other book in your life, this is the one
-
Kenneth Stanley reposted
gallorob
@gallorob2
Implementing safe mutations (a very interesting family of methods by @joelbot3000, Jay Chen, @jeffclune and @kenneth0stanley) in my thesis project took most of the day but I couldn't be happier now that it's working 😀
-
Kenneth Stanley reposted
Natasha Jaques
@natashajaques
How can we move deep RL beyond games, without having to hand-build a simulator that covers real-world complexity? We adversarially generate a curriculum of challenging yet feasible environments by maximizing regret between a pair of agents, with PAIRED... youtu.be/y_98BL-JxSM
-
Kenneth Stanley reposted
Sebastian Risi
@risi1979
After a few rejections happy this work w/ @kenneth0stanley is finally accepted and will appear at #aaa2021! Now just need to finally make the code pretty enough to be publicly releasable....
Sebastian Risi
@risi1979
Introducing Deep Innovation Protection - we scale neuroevolution to more complex tasks such as "VizDoom: Take Cover" by temporally reducing the selection pressure on specific components in an evolving world model. arxiv.org/abs/2001.01683 w/ @kenneth0stanley @UberAILabs @ITUkbh
-
Kenneth Stanley reposted
Mr. Eli W. Jones
@mreliwjones
"Why Open-Endedness Matters" by @kenneth0stanley
cognet.mit.edu/sites/default/files/artl_a_00294.pdf -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Our company modl.ai is looking for a team member passionate about building world class game engine tools (Unity / Unreal Engine) around AI/ML. Come work with us! More information:
modl-ai.breezy.hr/p/37092c02ab7001-lead-game-ai-tools-dev… -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
A collection of artificial life environments to study open-endedness. @kenneth0stanley @err_more @leespector @RELenski @CharlesOfria @ThomasMiconi @IMordatch Any major ones we forgot? Guess which framework will be next 🤓
-
Kenneth Stanley reposted
NAVER LABS Europe
@naverlabseurope
Seminar 📢'Why open-endedness matters to #ai' with
@kenneth0stanley @OpenAI Register today!
Date: Tue 10th Nov 5pm CET (GMT+1)
Register: www.eventbrite.com/e/naver-labs-europe-seminar-why-open-e…
#neuroevolution #neuralnetworks -
Kenneth Stanley
@kenneth0stanley
Great example of the problem with metric obsession. Creativity is not monotonic improvement.
-
Kenneth Stanley reposted
Lisa
@err_more
super excited! I'll be talking about evolution in the context of biology and technology + algorithms that discover new things. open to the public and there's a chat afterwards - feel free to join! 💙
CrossLabs
@crosslabstokyo
Excited to announce that our very own Dr. Lisa Soros @err_more will be presenting 'The Science of Innovation' on YouTube Live this Friday at 3pm JST. Tickets are free as always 🙌 www.eventbrite.com/e/cross-roads-17-the-science-of-innova… #ai #innovation
-
Kenneth Stanley
@kenneth0stanley
Thank you @VanceCrowe for the opportunity through your podcast to talk to a diverse audience about some of the inspirations and broader implications of our book beyond computer science and ML, among other wide-ranging topics! youtu.be/ipRAD8d6z5I
-
Kenneth Stanley reposted
Georgios N. Yannakakis
@yannakakis
From DeLeNox (image) in 2013 to MarioGAN and recent PCGML examples, deep learning has enriched the procedural generation of game content. We surveyed this exciting area of creative AI with @_JialinLiu @SamPSnodgrass @Amidos2006 @risi1979 and @togelius arxiv.org/pdf/2010.04548
Julian Togelius
@togelius
In the past few years, there has been an explosion of research on using deep learning for procedural content generation in games. @_JialinLiu @SamPSnodgrass @Amidos2006 @risi1979 @yannakakis and I chronicle this development in our new paper:
rdcu.be/b8gGZ -
Kenneth Stanley
@kenneth0stanley
Thank you @pgmid for the opportunity to connect ideas in AI to neuroscience and plenty of other wide-ranging topics as we spoke on this just-released Brain Inspired podcast! Had a great time!
Paul Middlebrooks
@pgmid
Are the best things created and discovered by pursuing objectives?
Nay, says @kenneth0stanley, the best things are made by open-ended processes run by novelty, creativity, intuition.
Ken wants to develop an infinitely open-ended algorithm.
Yea, says I!
braininspired.co/podcast/86/ -
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Nice meta-RL article (mentions POET and AI-Generating Algorithms). Learning to Learn More: Meta Reinforcement Learning by @rajatsahay719 in @TDataScience towardsdatascience.com/learning-to-learn-more-meta-reinfo… POET led by @ruiwang2uiuc w @joelbot3000, myself & @kenneth0stanley
-
Kenneth Stanley reposted
Max Jaderberg
@maxjaderberg
Latest surprising ICLR21 submission to stumble upon "Self-Organizing Intelligent Matter: A blueprint for an AI generating algorithm". Mesmerising emergent dynamics, each pixel contains an individual neural network. openreview.net/forum?id=160xFQdp7HR
This media couldn't be displayed. See it on the original post. -
Kenneth Stanley reposted
Kamal Ndousse
@kandouss
Big personal milestone! My first ML paper is on arXiv: arxiv.org/abs/2010.00581
We propose a simple method that helps RL agents in shared environments learn from one another, and show that the learned social policies improve zero-shot transfer performance in new environments. 👇 -
Kenneth Stanley
@kenneth0stanley
Great time reminiscing about the days of NEAT (and how things have changed since then) on @wellecks’ Thesis Review podcast! Also a great idea for a show, to look back years later with the authors of selected dissertations! Thank you @wellecks for having me.
The Thesis Review Podcast
@thesisreview
Episode 9 of The Thesis Review:
Kenneth Stanley (@kenneth0stanley), "Efficient Evolution of Neural Networks through Complexification"
We discuss neuroevolution and the NEAT algorithm, open-endedness and POET, and how the field has developed over time
soundcloud.com/thesis-review/09-kenneth-stanley-efficient… -
Kenneth Stanley
@kenneth0stanley
Intriguing demonstration of the potential of Hebbian plasticity in large networks at #neurips. Congrats @risi1979 and @enasmel!
Sebastian Risi
@risi1979
Congratulations to @enasmel for his (and my!) first accepted #neurips paper! 🎉
We added an experiment that shows that Hebbian networks can also sometimes generalize to robot morphologies not seen during training.
PDF: arxiv.org/pdf/2007.02686.pdf
Code: github.com/enajx/HebbianMetaLearning -
Kenneth Stanley reposted
Karl Cobbe
@karlcobbe
Excited to share our recent work on Phasic Policy Gradient, a new RL algorithm which improves sample efficiency by performing policy optimization and auxiliary optimization in two alternating phases. Check out the paper and code! arxiv.org/abs/2009.04416
-
Kenneth Stanley reposted
Jitka Cejkova
@CejkovaJitka
Book #robot100
Chapter "RUR and Deception" by @kenneth0stanley from @OpenAI (previously: @UberAILabs, @UCF)
What seems evident today can easily become eclipsed by unexpected deeper insight in the future.
www.cs.ucf.edu/~kstanley/
www.robot100.cz/home -
Kenneth Stanley reposted
Zoubin Ghahramani
@ZoubinGhahrama1
I’m delighted to announce that I’ve just joined Google Research as part of the leadership of the Brain team! In addition to my ongoing academic position, I’m really excited to now be part of @GoogleAI and its #machinelearning community!
-
Kenneth Stanley reposted
Antoine Cully
@CULLYAntoine
We are recruiting a Post-doc to join us on the REcoVER project (two years) at @imperialcollege . Please share this with anyone who might be interested: www.imperial.ac.uk/jobs/description/ENG01388/research-ass…
@ImperialAI @ImperialRobots @ICComputing
Antoine Cully
@CULLYAntoine
I thrilled to announce that my EPSRC project, REcoVER, has been accepted! It aims to push the state of the art of resilient and versatile robots. I am looking for a very talented post-doc to join this adventure for two years. I will publish the official job description soon.
-
Kenneth Stanley
@kenneth0stanley
Could not be more thrilled to be working together again with my amazing collaborator @joelbot3000, now at @OpenAI, and on so exciting a topic as open-endedness! Congrats to Joel on the new role and looking forward to reaching the next stepping stone together!
Joel Lehman
@joelbot3000
I’m excited to announce that I have joined @OpenAI as of last month, am happy to again team up with the singular @kenneth0stanley to push forward the frontiers of open-endedness. Unexpected where the stepping stones lead (image from 1st academic presentation at ALife 2008). 1/3
-
Kenneth Stanley reposted
Antoine Cully
@CULLYAntoine
Excellent question! Quality-Diversity optimisation algorithms and multi-objective optimisation may look similar, but they are actually quite different. Here a small thread:
Anthony Garland
@AnthonyPGarland
I read the tech article but not the journal paper. How is this different than say NSGA-II?
-
Kenneth Stanley
@kenneth0stanley
Thank you @vijayants for the sentiment- I don't think @joelbot3000 or I had realized the book might be worth more than one read! Glad to hear you get something new from it each time!
Baba Yoda
@theyodababa
Highly recommended book for most of us, everytime I re-read it (but of course, sans any objective or plan,) I get something new & marvellous... Today, it was the validation of an idea.
@kenneth0stanley & Joel Lehman. #computerscience #ai #life #books -
Kenneth Stanley reposted
GoodAI
@GoodAIdev
14 talks from the Meta-Learning & Multi-Agent Learning Workshop that took place last week are now online. Thank you to all of the amazing speakers and for the participants who also took part in the discussions and panels! www.youtube.com/playlist?list=PL9poXcROwgazgxVZvO4ZLBlH9D…
-
Kenneth Stanley
@kenneth0stanley
Thank you again to the organizers of the Meta-Learning & Multi-Agent Learning Workshop for the opportunity to talk (now available to watch below) on why open-endedness is important for AI and machine learning!
GoodAI
@GoodAIdev
The Importance of Open-Endedness in AI and Machine Learning a fascinating talk by Kenneth Stanley (@kenneth0stanley) of @OpenAI at the Meta-Learning & Multi-Agent Learning Workshop. youtu.be/AwlLrugFcOc
-
Kenneth Stanley
@kenneth0stanley
Thank you to the organizers of the Meta-Learning & Multi-Agent Learning Workshop for the opportunity to give a talk and to participate in your panel on open-endedness. That was a great discussion and inspiring to see so much interest and novel thinking on open-endedness!
GoodAI
@GoodAIdev
The Importance of Open-Endedness in AI & Machine Learning, Kenneth Stanley (@kenneth0stanley) of @OpenAI is currently giving a talk on open-endedness at the Meta-Learning & Multi-Agent Learning Workshop. Video available tomorrow. www.goodai.com/meta-learning-multi-agent-learning-worksho…
-
Kenneth Stanley reposted
GoodAI
@GoodAIdev
We will be joined online by @kenneth0stanley, who will be speaking on Open-Endedness in AI & ML, and many other exciting guests for the Meta-Learning & Multi-Agent Learning Workshop that starts next week! Check out the full schedule and reading list here: www.goodai.com/meta-learning-multi-agent-learning-worksho…
-
Kenneth Stanley
@kenneth0stanley
.@jbrant1983 & I changed the minimal criterion coevolution (MCC) open-ended alg to replace speciation with a simple resource limit with surprising benefit: MCC, now extremely simple, actually works better (& won Best Paper in Complex Systems @GeccoConf!) eplex.cs.ucf.edu/papers/brant_gecco20.pdf
-
Kenneth Stanley
@kenneth0stanley
Thanks to Jason Stoughton @ThePulseOfAI podcast for the excellent conversation with @babakatwork on Creativity and Open-Endedness in AI! Very much enjoyed the discussion: www.thepulseofai.com/post/creativity-and-open-endedness-i…
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
I'm giving a talk today at the ICML Continual Learning Workshop: "Learning to Continually Learn" introduces AI-GAs as a path to AGI and focuses on ANML as an example of that research paradigm that works well to minimize catastrophic forgetting. icml.cc/Conferences/2020/ScheduleMultitrack?event=5743 @icmlconf
-
Kenneth Stanley
@kenneth0stanley
Great to see attempts to built on the POET concept!
Julian Togelius
@togelius
Does it seem familiar somehow? Yes, indeed, we build on the POET algorithm by @ruiwang2uiuc @joelbot3000 @kenneth0stanley @jeffclune. But where POET created new simple walker environments, we create complete levels for games with comparatively complex rules.
-
Kenneth Stanley reposted
Jeff Dean
@JeffDean
AutoML-Zero: new research from @GoogleAI researchers Esteban Real, @crazydonkey200, David R. So, & @quocleix that that can rediscover fundamental ML techniques by searching a space of different ways of combining basic mathematical operations.
Arxiv: arxiv.org/abs/2003.03384
Google AI
@GoogleAI
Extending research into evolutionary #automl, we present an approach that evolves algorithms from scratch—using only basic mathematical operations—rediscovering fundamental ML techniques & showing the potential to discover novel algorithms. Read more ↓ goo.gle/3edHPBc
-
Kenneth Stanley reposted
ML and Data Projects To Know
@PTKDaily
🖥️ Fiber by @_calio, @ruiwang2uiuc, @jeffclune, @kenneth0stanley
Distributed Computing for AI Made Simple
Featured in PTK #43 mailchi.mp/amplifypartners.com/ptk43
Code: github.com/uber/fiber
Paper: arxiv.org/abs/2003.11164
Website: uber.github.io/fiber/ -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
@enasmel and myself are excited to announce our paper "Meta-Learning through Hebbian Plasticity in Random Networks" arxiv.org/abs/2007.02686
Instead of optimizing the neural network's weights directly, we only search for synapse-specific Hebbian learning rules. Thread 👇 -
Kenneth Stanley reposted
Rui Wang
@ruiwang2uiuc
We (w/ @_calio) @UberEng are excited to give a @GeccoConf Tutorial (July 9, 2pm PDT) “A Hands-on Guide to Distributed Computing Paradigms for EC & RL”. Featuring Fiber @UberOpenSource!
Thank @kenneth0stanley @jeffclune @joelbot3000 @joost_Huizinga @AdityaRawaI for advices & help! -
Kenneth Stanley
@kenneth0stanley
Happy to share that our paper "Enhanced POET: Open-ended RL through Unbounded Invention of Learning Challenges & their Solutions" will appear at #icml2020 arxiv.org/abs/2003.08536 Led by @ruiwang2uiuc w/ @joelbot3000, @AdityaRawaI, @_calio, Yulun Li, @jeffclune, and myself 1/2
Kenneth Stanley
@kenneth0stanley
Blog: eng.uber.com/enhanced-poet-machine-learning
Code: github.com/uber-research/poet 2/2 -
Kenneth Stanley
@kenneth0stanley
Generative Teaching Networks accepted at ICML! blog: eng.uber.com/generative-teaching-networks paper: arxiv.org/abs/1912.07768 GTNs learn to generate data/experience to teach other networks! Led by @felipesuch w @AdityaRawaI @joelbot3000 @jeffclune
-
Kenneth Stanley reposted
Joseph Suarez 🐡
@jsuarez
Neural MMO v1.4 is live -- Quality baselines + in-game attention/value/etc overlays + RLlib compatibility!
New and improved Documentation: jsuarez5341.github.io
Community Discord: discord.gg/BkMmFUC
Github: github.com/jsuarez5341/neural-mmo
Demo: youtu.be/y_f77u9vlLQThis media couldn't be displayed. See it on the original post. -
Kenneth Stanley
@kenneth0stanley
Excited to announce positions available in the Open-Endedness Team I'm starting @OpenAI: Research Engineer jobs.lever.co/openai/4ffb6cac-e2e6-4b9e-878a-0425e1463f3c Research Scientist jobs.lever.co/openai/d61405f0-6172-4822-8fca-2bfaaf1e2e69 - looking forward to exploring new frontiers! 1/2
Kenneth Stanley
@kenneth0stanley
For all OpenAI jobs see openai.com/jobs/ 2/2
-
Kenneth Stanley
@kenneth0stanley
A blog post on Fiber: a platform designed to make #distributedcomputing for AI easy. Program a whole cluster with thousands of workers with just the standard multiprocessing API. (Enhanced POET used Fiber!) eng.uber.com/fiberdistributed/ led by @_calio w/ @ruiwang2uiuc @jeffclune
-
Kenneth Stanley
@kenneth0stanley
Thank you @MarwaEldiwiny for the very stimulating conversation on the IEEE RAS Robotics Podcast! It was a pleasure to join your show!
Marwa ElDiwiny
@MarwaEldiwiny
Compelling discourse with Kenneth Stanley @kenneth0stanley Open-endedness Group Leader @OpenAI "How we can reach Open-endedness AI & Why greatness cannot be planned" What are the real challenges, I think listening to Kenneth is inspiring, I hope you enjoy
soundcloud.com/ieeeras-softrobotics/kenneth-stanley-open-… -
Kenneth Stanley
@kenneth0stanley
Congratulations @_joelsimon on such a creative and socially relevant application of concepts from novelty search and quality diversity. I hope this has the impact on our democracy that it deserves to. Very inspiring work!
Joel Simon
@_joelsimon
Excited to share antimander.org 💻🇺🇸
A paper, interactive blog, and open-source initiative to help expose gerrymandering by generating many optimized alternatives. Motivated by how generative design can make complex tradeoffs understandable. 1/ -
Kenneth Stanley reposted
OpenAI
@OpenAI
We're releasing an API for accessing new AI models developed by OpenAI. You can "program" the API in natural language with just a few examples of your task. See how companies are using the API today, or join our waitlist: beta.openai.com
-
Kenneth Stanley
@kenneth0stanley
Announcing Synthetic Petri Dish for architecture search: An NN slice goes into a "dish" where tiny synthetic data is learned to make variants perform like the full NN. Then rapid search in the dish: beyond a model because it IS part of the real NN arxiv.org/pdf/2005.13092.pdf 1/2
Kenneth Stanley
@kenneth0stanley
Led by @AdityaRawaI with @joelbot3000 @felipesuch and @jeffclune - a fantastic team of collaborators! code at github.com/uber-research/Synthetic-Petri-Dish 2/2
-
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Time flies! My first paper as a PhD student in 2009 with @kenneth0stanley investigated deceptiveness in meta-learning tasks. e.g. an agent has to learn to remember the position of a high reward in a T-maze environment. What do I mean with deceptiveness? 👇
-
Kenneth Stanley
@kenneth0stanley
In case you didn’t have time for our arxiv paper on Enhanced POET, a quicker blog post with videos and pictures was just released. See enhancements like domain-general environmental diversity metrics and a new open-ended progress measure: eng.uber.com/enhanced-poet-machine-learning 1/
Kenneth Stanley
@kenneth0stanley
Work led by @ruiwang2uiuc w/ @joelbot3000, @AdityaRawaI, @_calio, Yulun Li, @jeffclune, and myself. Arxiv: arxiv.org/abs/2003.08536 2/2
-
Kenneth Stanley reposted
Sam Kriegman
@Kriegmerica
Are your virtual creatures out of shape?
Start training them now so they’ll be ready in 8 weeks for the 2020 Virtual Creatures Competition
virtualcreatures.github.io
held virtually at @ALifeConf.
Submission deadline: June 26. -
Kenneth Stanley reposted
Jake Scott
@CancerConnector
I just finished a great book: "Why Greatness Cannot Be Planned: The Myth of the Objective". It is a book about discovery, curiosity and novelty written by a pair of computer scientists. a THREAD with some polls... (1/n) link.springer.com/content/pdf/10.1007/978-3-319-15524-1.pdf
-
Kenneth Stanley reposted
Pierre-Yves Oudeyer
@pyoudeyer
This work leverages the intrinsically motivated goal
exploration framework, adding incrementally learned goal
spaces.
It also uses the Lenia #gameoflife model from @BertChakovsky and
CPPNs from @kenneth0stanley to parameterize initial states. -
Kenneth Stanley
@kenneth0stanley
Major advances in Go-Explore: far more SoTA results, superhuman across Atari, stochastic training, hard-exploration simulated robotics. pdf: arxiv.org/abs/2004.12919 & code updated soon: github.com/uber-research/go-explore. with @AdrienLE (lead) @Joost_Huizinga (lead) @joelbot3000 @jeffclune
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Here is the final link to my talk at the ICLR BeTR-RL workshop on AI-GAs and Learning to Continually Learn (with ANML). slideslive.com/38926301 Thanks to the workshop organizers and my collaborators! #iclr2020 #iclr
-
Kenneth Stanley
@kenneth0stanley
Had a great time speaking with @bigdata on #thedataexchangepod on open-endedness, machine learning, and my future plans. Thanks Ben!
Ben Lorica 罗瑞卡
@bigdata
Thread: In this episode of #thedataexchangepod I speak with @kenneth0stanley, a Senior Research Manager @UberAILabs and a Professor @UCFComSci 🆕📰 Breaking News: Ken has accepted a position @OpenAI & starts there in June 🎊 thedataexchange.media/algorithms-that-continually-invent-…
-
Kenneth Stanley
@kenneth0stanley
I'm thrilled to announce that I will be joining the superb team at @OpenAI in June, where I will be starting a group (and indeed hiring) focused on achieving open-endedness in machine learning. Looking forward to exploring a novel path!
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Source code is now available for Generative Teaching Networks eng.uber.com/generative-teaching-networks Code github.com/uber-research/GTN Courtesy of @felipesuch's amazing research & engineering. We hope this helps the community. Let us know if you use it! w/ @kenneth0stanley @joelbot3000 @AdityaRawaI
-
Kenneth Stanley reposted
Rui Wang
@ruiwang2uiuc
Excited to announce: source code for Enhanced POET now available at github.com/uber-research/poet
paper: arxiv.org/abs/2003.08536
w/ @joelbot3000 @AdityaRawaI @_calio Yulun Li
@jeffclune and @kenneth0stanley
Distributed computing in our code enabled by Fiber: uber.github.io/fiber/ -
Kenneth Stanley
@kenneth0stanley
Pleased to announce Fiber, a platform designed to make #distributedcomputing for AI easy. Program a whole cluster with thousands of workers with just the standard multiprocessing API. (Enhanced POET used Fiber!) uber.github.io/fiber/introduction/
Kenneth Stanley
@kenneth0stanley
Fiber owes its existence mainly to the incredible efforts of @_calio , with support and input from excellent colleagues @ruiwang2uiuc and @jeffclune
-
Kenneth Stanley reposted
Vashisht Madhavan
@vashmadhavan
FIber is really slick! Makes distributed training/inference as easy as using python multiprocessing. Great job getting this out @_calio and team!
Jiale Zhi
@_calio
Introducing Fiber, a new framework that made #distributedcomputing for AI simple. Now you can program the whole computer cluster and running thousands of workers easily with the standard multiprocessing API that you are familiar with.uber.github.io/fiber/introduction/ 1/2
-
Kenneth Stanley
@kenneth0stanley
Excited to announce a major enhancement of the POET algorithm - broadening and simplifying its potential application to make almost any conceivable domain open-ended. With @ruiwang2uiuc @joelbot3000 @jeffclune @AdityaRawaI @_calio Yulun Li and @jeffclune arxiv.org/abs/2003.08536
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Open source code now available for ANML github.com/uvm-neurobotics-lab/ANML From Learning to Continually Learn (Beaulieu et al. ECAI 2020). ANML meta-learns how to learn without catastrophic forgetting, including up to 600 sequential tasks. Talk: videos.re-work.co/videos/1861-anml-learning-to-continuall…
-
Kenneth Stanley
@kenneth0stanley
If every student in the country/world starts attending classes online, does that pose any problem for the internet locally or globally?
-
Kenneth Stanley reposted
Quoc Le
@quocleix
Fun AutoML-Zero experiments: Evolutionary search discovers fundamental ML algorithms from scratch, e.g., small neural nets with backprop.
Can evolution be the “Master Algorithm”? ;)
Paper: arxiv.org/abs/2003.03384
Code: git.io/JvKrZ -
Kenneth Stanley reposted
Vashisht Madhavan
@vashmadhavan
New paper! Scaling MAP-Elites to Deep Neuroevolution. We enable agents to recover from damage and explore in high-D control tasks, where traditional QD algorithms fail. Arxiv arxiv.org/abs/2003.01825. Work led by @cedcolas during his internship with me @UberAILabs 1/3
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Learning to Continuously Learn. ANML meta-learns to reduce catastrophic forgetting, and can learn at least 600 tasks (Omniglot classes) sequentially and performs well on most afterwards. arXiv arxiv.org/abs/2002.09571 NeurIPS talk slideslive.com/38924020/how-alternatives-to-conventional-… Led by Shawn Beaulieu 1/2
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Work done with a great team: Shawn Beaulieu, Lapo Frati, @ThomasMiconi, Joel Lehman (@joelbot3000), @kenneth0stanley, and Nick Cheney. ANML adds to my belief that we should pursue AI-generating algorithms and meta-learn the solutions to the hardest machine learning challenges 2/2
-
Kenneth Stanley reposted
hardmaru
@hardmaru
HyperNEAT is quite funky. eplex.cs.ucf.edu/hyperNEATpage/HyperNEAT.html
Smerity
@Smerity
What's the weirdest neural architecture you've seen (or might imagine) that's not the standard convention?
Bonus points if it uses neglected ideas / hardware ;)
(think Neural GPU, evolution strategies, Weight Agnostic Neural Networks, Beowulf cluster of Tamagotchis, ..?) -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Maybe even Evolvable-substrate HyperNEAT-LENIA!🕸️ES is already incrementally growing neural networks and a little more robust in translating weight patterns into functional networks than pure HyperNEAT eplex.cs.ucf.edu/ESHyperNEAT/
-
Kenneth Stanley reposted
Alex Spurrier
@alspur
This reminds me of one of my favorite 📚 of the decade, Why Greatness Cannot Be Planned by @kenneth0stanley.
The decision tree of life is foggy and the “right” path can’t be known. Don’t plan a route, set a *direction* and re-calibrate frequently.
www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/… -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Introducing Deep Innovation Protection - we scale neuroevolution to more complex tasks such as "VizDoom: Take Cover" by temporally reducing the selection pressure on specific components in an evolving world model. arxiv.org/abs/2001.01683 w/ @kenneth0stanley @UberAILabs @ITUkbh
-
Kenneth Stanley
@kenneth0stanley
Despite at least one cause being apparent (less data on faces on the periphery), it's still interesting to see the train-wreck when a photo-realistic face-trained GAN makes the unfortunate choice to place a "friend" beside the main subject... (more in reply)
Kenneth Stanley
@kenneth0stanley
Life is not easy when you're posing next to a GAN-generated face.
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Wow. 24 hours from our blog post to someone posting a web tutorial/summary of our Generative Teaching Networks! Impressively fast! www.youtube.com/watch?v=lmnJfLjDVrI work led by @felipesuch with @AdityaRawaI @joelbot3000 and @kenneth0stanley
-
Kenneth Stanley
@kenneth0stanley
Introducing Generative Teaching Networks - they generate synthetic data optimized to train learners as fast as possible - a new way to generate data for architecture search & more! eng.uber.com/generative-teaching-networks/ Led by @felipesuch w/ @jeffclune, @joelbot3000, & @AdityaRawal
-
Kenneth Stanley reposted
Robert Levy
@rplevy
There's definitely a common philosophical thread running through @paulg's latest essay www.paulgraham.com/lesson.html and @kenneth0stanley and Joel Lehman's "Why Greatness Cannot Be Planned" www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/…
-
Kenneth Stanley reposted
Antoine Cully
@CULLYAntoine
If you are willing to learn more about Quality-Diversity algorithms, we have created a website that gathers papers, tutorials, and implementations of QD algorithms.
The progressive increase of papers published on this topic every year is impressive!
quality-diversity.github.io -
Kenneth Stanley reposted
Bill Rand
@billrand
Great discussion on lifelong learning in Neural Networks, highlights work by @jeffclune and @kenneth0stanley. Lifelong learning was a notion I wanted to study early on.
#lifelonglearning #ai #machinelearning #plasticity
cacm.acm.org/magazines/2019/6/236992-lifelong-learning-in… -
Kenneth Stanley reposted
Julian Togelius
@togelius
CMA-ME (that's the acronym) builds on the ideas of quality diversity algorithms as pioneered by @kenneth0stanley @joelbot3000 @err_more, in particular the MAP-Elites algorithm by @jeffclune @jb_mouret @CULLYAntoine, and combines this with principles of modern optimization.
-
Kenneth Stanley reposted
Sam Charrington
@samcharrington
Great listening on the topic for those interested: My @twimlai convos with @kenneth0stanley and Risto Miikkulainen
twimlai.com/twiml-talk-94-neuroevolution-evolving-novel-n…
twimlai.com/twiml-talk-47-evolutionary-algorithms-machine… -
Kenneth Stanley reposted
Quanta Magazine
@QuantaMagazine
To create better AI, researchers are attempting to emulate the only system we’ve seen develop anything remotely like human intelligence: biological evolution.
www.quantamagazine.org/computers-evolve-a-new-path-toward… -
Kenneth Stanley
@kenneth0stanley
Nice coverage by @SilverJacket of some of the big ideas behind our work and some of the history of neuroevolution for @QuantaMagazine at quantamagazine.org/computers-evolve-a-new-path-toward-hum… ...thanks for the thoughtful article!
-
Kenneth Stanley reposted
Bert Chan
@BertChakovsky
Lenia is now used in machine learning research!
by @ScienceReinke, M Etcheverry & @pyoudeyer. They used curiosity-driven IMGEP algorithm with CPPN-NEAT and VAE to auto discover artificial life-forms, with interesting results (increased diversity). arxiv.org/abs/1908.06663 -
Kenneth Stanley reposted
O'Reilly Media
@OReillyMedia
Open-ended algorithms can offer an entirely different level of automated creation. @kenneth0stanley talks more about this new grand challenge for #ai at the @OReillyAI conference in San Jose. oreil.ly/FDGBE
-
Kenneth Stanley reposted
Erlend Ekern
@erlendekern
Infinite-resolution image generation in the browser using CPPNs, GANs and some TensorFlow.js wizardy 🧙 Check it out LIVE at
🚀🚀🚀thispicturedoesnotexist.com🚀🚀🚀 ((following the naming convention of GAN websites, of course 😄))) @hardmaru @kenneth0stanley -
Kenneth Stanley reposted
hardmaru
@hardmaru
Evolving Networks: A nice review article about NEAT and extensions, with a minimal python implementation.
Interesting to read about Neuroevolution from people like @wellecks who mainly work on Deep Learning and NLP.
wellecks.wordpress.com/2019/07/21/evolving-networks/ -
Kenneth Stanley reposted
OReillyAI
@OReillyAI
@kenneth0stanley Sr Research Scientist @UberAILabs is one of the many expert keynote speakers at the #oreillyai conference. Final weeks to register! Save your spot today. oreil.ly/2Wz9pUK
-
Kenneth Stanley reposted
Uber Engineering
@UberEng
We are very proud that @Uber AI Labs' Rui Wang, Joel Lehman, Jeff Clune, and Ken Stanley won a Best Paper Award @GECCO2019 for POET, the ML algorithm released earlier this year!
More about POET: ubere.ng/2RfKsvr
Read the paper: ubere.ng/2H4nH94
#ml #ai #deeplearning -
Kenneth Stanley reposted
Erlend Ekern
@erlendekern
A deep CPPN trained on 64x64 celebrity images and generating new samples at 150 times the resolution of the training data 🚀 Interpolation between different latent vectors 👇@hardmaru @kenneth0stanley #generative #cppn #gan
-
Kenneth Stanley
@kenneth0stanley
This EvoGrad library (and its current Evolvability ES use case) opens up really interesting new opportunities for the application of deep neuroevolution at scale. It's worth a look!
Alex Gajewski
@apagajewski
Announcing our EvoGrad library for grad-based evolution + our Evolvability ES meta-learning algorithm: can scale to deep nets, compete w/ MAML in RL. With @jeffclune, @kenneth0stanley, and @joelbot3000. Blog: eng.uber.com/evograd, paper: arxiv.org/abs/1907.06077
-
Kenneth Stanley
@kenneth0stanley
Monday 11:05am at GECCO, Jonathan Brant presents our new paper, "Benchmarking Open-Endedness in Minimal Criterion Coevolution". paper here: eplex.cs.ucf.edu/papers/brant_gecco19.pdf Hoping its low-compute open-source maze benchmark github.com/jbrant/MinimalCriterionCoevolution/releases/ta… enables a new wave of OEE investigations!
-
Kenneth Stanley reposted
Erlend Ekern
@erlendekern
Interpolation between different latent vectors of a CPPN trained on the CelebA dataset. The visual properties are determined by the network structure and activation functions (LReLU and tanh, in this case). Any familiar faces in there? 👀 @hardmaru
-
Kenneth Stanley reposted
Julian Togelius
@togelius
Quality-diversity algorithms are an important new class of search algorithms. They open up new possibilities in games, in particular for procedural content generation. In our new paper, we survey early applications of quality-diversity to PCG, and lay out a path forward.
Daniele Gravina
@gravinadan
How to produce a large set of diverse and high-quality artifacts for games? We propose procedural content generation through quality-diversity (PCG-QD) as a subset of search-based procedural content generation
arxiv.org/abs/1907.04053 -
Kenneth Stanley
@kenneth0stanley
Video of our ICML tutorial (with @jeffclune and @joelbot3000 ) covering topics including Quality Diversity (first), Open-Endedness (1:14:20), and Indirect Encoding (1:51:50) is at youtu.be/g6HiuEnbwJE . If you haven't heard of these topics, they're worth a look!
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
Slides now available for our ICML Tutorial on Population-Based Methods for Training Deep Neural Networks: Novelty Search, Quality Diversity, Open-Ended Search Algorithms, & Indirect Encoding. w @kenneth0stanley & @joelbot3000 #icml2019 Video soon. www.cs.uwyo.edu/~jeffclune/share/2019_06_10_ICML_Tutorial…
-
Kenneth Stanley
@kenneth0stanley
Novelty search, quality diversity (QD), open-endedness, and indirect encoding, all in an #icml2019 tutorial on Monday morning! Excited to be presenting with @jeffclune and @joelbot3000 : icml.cc/Conferences/2019/Schedule?showEvent=4336
-
Kenneth Stanley reposted
OReillyAI
@OReillyAI
.@kenneth0stanley Sr Research Scientist @UberAILabs is one of the many expert keynote speakers at the #oreillyai conference. Register before June 14 for Best Price! oreil.ly/2Wz9pUK
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence arxiv.org/abs/1905.10985
Historically, hand-designed pipelines are ultimately outperformed by entirely learned ones. Will that will be true of creating general AI itself? 1/6 -
Kenneth Stanley reposted
Rui Wang
@ruiwang2uiuc
Thanks! Had a great night @LinkedIn !
BTW, we’ve also released source code of POET at github.com/uber-research/poet
Work is done w/ @joelbot3000 @jeffclune @kenneth0stanley
Sergii 🇺🇦
@lc0d3r
Great talk by Rui Wang @ruiwang2uiuc from @UberAILabs on Open-Ended
Reinforcement Learning - Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions
arxiv.org/abs/1901.01753 or eng.uber.com/poet-open-ended-deep-learning/ -
Kenneth Stanley reposted
Tim Taylor
@drtimt
I just prepared and uploaded a shareable version of our editorial introduction to the first special issue on open-ended evolution in the #artificiallife journal. Available at www.researchgate.net/publication/332132174_Open-Ended_Evo… #openaccess @nhpackard @alltbl @kenneth0stanley
-
Kenneth Stanley reposted
Thomas Miconi
@ThomasMiconi
Nature Machine Intelligence (@NatMachIntell) has just published a News&Views article by @tyrell_turing about our Backpropamine framework (differentiable, trainable neuromodulation of plasticity in neural networks)!
Read it there:
rdcu.be/bzByv -
Kenneth Stanley reposted
Sebastian Risi
@risi1979
Happy to announce our new @GECCO2019 paper on "Deep Neuroevolution of Recurrent and Discrete World Models" with @kenneth0stanley. A simple GA is able to learn the model end-to-end and finds effective representations for the visual and memory component.
-
Kenneth Stanley reposted
Joel Simon
@_joelsimon
Generative architecture project from last few months. Custom generator / multi objective optimization / novelty search / exploration viewer. Still needs some work.
-
Kenneth Stanley
@kenneth0stanley
Consider submitting to the Exploration in RL workshop at ICML: sites.google.com/view/erl-2019 . It's open to topics like novelty search, QD, and open-endedness, plus more conventional RL topics - a unique combination!
-
Kenneth Stanley reposted
Piero Molino
@w4nderlus7
Ludwig v0.1.1 is out! github.com/uber/ludwig/re… @Cometml integration, @TensorFlow 1.13.1 support, multilanguage support thanks to @spacy_io 2.1, new parameters and many bugfixes. Enjoy! @UberEng
-
Kenneth Stanley reposted
Rui Wang
@ruiwang2uiuc
Source code is now available for POET! It generates its own increasingly complex, diverse training environments & solves them, auto-generating curricula along the way. github.com/uber-research/poet With @jeffclune @joelbot3000 & @kenneth0stanley Blog @UberEng : eng.uber.com/poet-open-ended-deep-learning
-
Kenneth Stanley reposted
ISAL Artificial Life
@alifeofficial
The first of a two-part special issue of the Artificial Life journal on Open-Ended Evolution is now out! www.mitpressjournals.org/toc/artl/25/1
-
Kenneth Stanley
@kenneth0stanley
Proud to have two of Re-Work's 30 under 30 women who are rising stars in #ai at Uber AI! Congrats to Rosanne Liu @savvyRL and Ashley Edwards @RealAshEdwards !
RE•WORK
@teamrework
Happy #internationalwomensday! We’re celebrating by looking at some of the rising stars in #ai - take a look at who’s inspiring us in #ai - www.re-work.co/blog/30-under-30-celebrating-rising-stars-…
Who’s your female inspiration?
#iwd2019
#womenintech
#womeninstem
#balanceforbetter
#happywomensday2019 -
Kenneth Stanley reposted
Piero Molino
@w4nderlus7
Almost two years of my work went into this, and now you can enjoy it too :)
Uber Engineering
@UberEng
Out today: Uber #ai introduces Ludwig, an #opensource, #deeplearning toolbox built on top of @TensorFlow that allows users to train and test deep learning models without writing code.
Read the article: ubere.ng/2I8oztY
Check out the Ludwig repo: uber.github.io/ludwig/ -
Kenneth Stanley reposted
Adrien Ecoffet
@AdrienLE
Go-explore source code is now available! github.com/uber-research/go-explore
Paper: arxiv.org/abs/1901.10995 Blog: eng.uber.com/go-explore/ @Joost_Huizinga @joelbot3000 @jeffclune @kenneth0stanley github.com/uber-research/go-explore -
Kenneth Stanley reposted
Julian Togelius
@togelius
Actually, AlphaStar is an evolutionary algorithm. In particular, a qualiyy-diversity algorithm with elements of competitive coevolution. A thinkpiece by @KaiLashArul with help from @CULLYAntoine and me
Kai Arulkumaran
@kaixhin
Hot take to hot off the press -
AlphaStar: An Evolutionary Computation Perspective w/ @CULLYAntoine @togelius: arxiv.org/abs/1902.01724 -
Kenneth Stanley reposted
Adrien Ecoffet
@AdrienLE
Go-Explore paper out. New high/average scores: 18 million/650k on Montezuma's Revenge (44k w/o domain knowledge) & 100k+/~60k on Pitfall, all tested w/ sticky actions! Huge thanks to my co-authors @Joost_Huizinga @joelbot3000 @jeffclune @kenneth0stanley. arxiv.org/abs/1901.10995
-
Kenneth Stanley
@kenneth0stanley
The Paired Open Ended Trailblazer (POET) algorithm (described at eng.uber.com/poet-open-ended-deep-learning/) was covered in Science by Matthew Hutson @SilverJacket ! www.sciencemag.org/news/2019/01/artificial-intelligence-t… (project led by @ruiwang2uiuc w/ @joelbot3000 @jeffclune )
-
Kenneth Stanley reposted
Jeff Clune
@jeffclune
A nice, short Scientific American about Thomas Miconi's (@ThomasMiconi ) differential plasticity work. www.scientificamerican.com/article/artificial-intelligenc… ICML paper proceedings.mlr.press/v80/miconi18a.html w/ @kenneth0stanley
-
Kenneth Stanley
@kenneth0stanley
Thank you @_cturner for this flattering analogy on our book! (adding my coauthor @joelbot3000 )
Chris Turner
@_cturner
Why Greatness Cannot Be Planned by @kenneth0stanley is an amazingly original and thought-provoking book that relatively few people know about. Got me thinking - if Thinking Fast and Slow is the nonfiction equivalent of Led Zeppelin's Stairway to Heaven, what are the deep tracks?
-
Kenneth Stanley
@kenneth0stanley
Just published through Open Access at Nature Machine Intelligence: a review of promising ideas and connections to deep learning from neuroevolution: “Designing neural networks through neuroevolution” www.nature.com/articles/s42256-018-0006-z?WT.feed_name=su… w/ @jeffclune @joelbot3000 & Risto Miikkulainen
-
Kenneth Stanley reposted
Joel Lehman
@joelbot3000
Happy to announce the Atari Zoo -- a collection of trained reinforcement learning agents and open-source software to help catalyze their understanding: eng.uber.com/atari-zoo-deep-reinforcement-learning/
w/ many great collaborators at UberAI, Google Brain, and OpenAI -
Kenneth Stanley
@kenneth0stanley
Great experience working with @ruiwang2uiuc on this project, who did an amazing job engineering the first system of its kind.
Rui Wang
@ruiwang2uiuc
We @UberAILabs announce POET, the Paired Open-Ended Trailblazer algorithm that continually generates its own increasingly complex & diverse learning environments and their solutions. w/ @joelbot3000 @jeffclune @kenneth0stanley eng.uber.com/poet-open-ended-deep-learning youtu.be/D1WWhQY9N4g
-
Kenneth Stanley
@kenneth0stanley
Pleased to be a part of a fascinating collection of interviews by @SentientDAI on the role of #evolutionaryalgorithms in the future of #ai. www.sentient.ai/labs/experts/





























































































































































































































































































































