How can we make any LLM produce near unbounded length outputs while fulfilling complex user-specified requirements from a single short prompt?!?
We introduce the LLM Automatic Computer (L2MAC) framework, which is state-of-the-art for large codebase generation tasks and can even generate entire books! From our new paper just published at ICLR 2024: openreview.net/pdf?id=EhrzQwsV4K and open-sourced: github.com/samholt/l2mac; which is easy to use.
L2MAC's benefits include:
ā
Strong empirical performance, state-of-the-art for large codebase generation tasks, and achieves a 90.2% Pass@1 score on HumanEval, currently globally ranked 4th: paperswithcode.com/sota/code-generation-on-humaneval
ā
Can generate near unbounded length outputs, significantly bypassing the finite context window token limit, and supports any LLM.
ā
Can produce detailed outputs that fulfill complex user-specified requirements.
ā
Automatically error corrects generated outputs, such as code that has syntactic and functional errors.
ā
Correctly conditions on and integrates with the existing previously generated outputs.
L2MAC is the first practical LLM-based general-purpose stored-program automatic computer (von Neumann architecture) framework, an LLM-based multi-agent system, for long and consistent output generation.
Its memory has two components: the instruction registry, which is populated with a prompt program to solve the user-given task, and a file store, which will contain the final and intermediate outputs. Each instruction, in turn, is executed by a separate LLM agent, whose context is managed by a finite state machine control unit capable of precise memory reading and writing to ensure effective interaction with the entire file store.
Check out more:
āļø Code: github.com/samholt/l2mac
š Online Documentation: samholt.github.io/L2MAC/
š° Paper: openreview.net/pdf?id=EhrzQwsV4K; with @MaxRuizLuyten and @MihaelaVDS
š¼ļø Gallery of output examples: samholt.github.io/L2MAC/guide/use_cases/gallery.html
#ai #multiagentllm #codegeneration #llms #iclr2024