Politicians, athletes, journalists, NGOs, companies... They all use
corporation-owned social networks to communicate with us. Citizens
shouldn't have to hand over their data to companies to read content
published by public entities. Public Feeds changes that by making posts
of popular accounts freely available.
Read more.
Christopher Ré
Christopher Ré is an American computer scientist. He is currently employed by Stanford University, where he is a full professor. He was awarded a MacArthur Fellowship in 2015. Ré specializes in big data analysis. He co-founded Lattice.io, a data mining and machine learning company that was acquired by Apple in May 2017. More on Wikipedia
Last Year We Built Megakernels. This Year We Deleted Them.
One year ago, we helped introduce some of the first AI megakernels to the world. Since then, we’ve enjoyed seeing their wide adoption from friends and researchers. However, while whole model megakernels are a helpful abstraction, we’ve found that they overcomplicate and are not always an ideal optimization target.
Below, we discuss our journey to understand and push past these limitations. We start with a deep dive into two contrasting paradigms of ML optimization – Torch Compile and Megakernels
Two weeks ago we announced Together Compiler: agents that tune inference to your model, chip, and traffic, for up to 2.6x more throughput.
Today, we’re sharing the journey behind it: the experiments that challenged our assumptions of megakernels, the surprising strengths of Torch Compile, and the lessons that shaped our development of a next-generation system for inference optimization.
Last Year We Helped Build Megakernels. 🔨 This Year We Deleted Them. ❌
Check out our post and new blogs for a deepdive into how we built the Together Compiler and the lessons learned along the way!
Some key takeaways: 1. When tuned appropriately, simply using Torch Inductor performs as well as hand-written Megakernels on canonical Megakernel workloads. 2. Full-model Megakernels are the wrong target to blindly hill-climb on. Fusion can be harmful. 3. The optimal abstraction for inference optimization is neither individual kernels nor one giant megakernel. And a bunch of interesting lessons on how agents interacted with various scaffoldings, compiler intermediate representations, and "expert guidance" we baked into the harness.
Last Year We Built Megakernels. This Year We Deleted Them.
One year ago, we helped introduce some of the first AI megakernels to the world. Since then, we’ve enjoyed seeing their wide adoption from friends and researchers. However, while whole model megakernels are a helpful abstraction, we’ve found that they overcomplicate and are not always an ideal optimization target.
Below, we discuss our journey to understand and push past these limitations. We start with a deep dive into two contrasting paradigms of ML optimization – Torch Compile and Megakernels
Two weeks ago we announced Together Compiler: agents that tune inference to your model, chip, and traffic, for up to 2.6x more throughput.
Today, we’re sharing the journey behind it: the experiments that challenged our assumptions of megakernels, the surprising strengths of Torch Compile, and the lessons that shaped our development of a next-generation system for inference optimization.
We created the Revenue Agent category to help the world's most consequential enterprises grow revenue with agents.
Today, our customers have closed over $7.5 billion in revenue since partnering with Rox.
Our core is the Global 2000: dozens of large public companies across medical devices, manufacturing and financial services.
CEOs are using Rox and rolling it our across their revenue teams. We’re building AI for the real economy, not just tech.
Since launch:
- ARR up 10x - 25x customer ROI within months - 700%+ NRR across enterprise accounts
So what is a Revenue Agent?
It’s an AI agent that finds, wins and grows every revenue opportunity for you behind the scenes.
Soon you’ll tell Rox “Generate $50M in opportunities” and it will figure out how.
In practice:
- Fortune 500 medical devices company: Rox finds rural hospitals with new funding and preps everything reps need to win, down to the joint funding proposal. That’s $200M in pipeline and $24M closed-won in months.
- Top-3 US payroll provider: Rox reaches millions of small businesses, like your local pizza shop. Reps just show up to the meeting, and winning a customer now costs 80% less.
- Global IT services business: $300M+ in qualified pipeline in 8 weeks.
All of this is built on our elite applied AI team of ~50 researchers in SF. Nobody else has a team like it.
Gartner named us the only Visionary in the 2026 Gartner® Magic Quadrant™ for Revenue Action Orchestration.
We’re the fastest-growing company in GTM, with the highest ROI of any AI spend.
With this round, we’re doubling down on our mission:
Thrilled to announce that our @GoogleDeepMind team has cracked a major challenge in software engineering and cybersecurity: formally verifying a real-world implementation of the #signal cryptographic protocol at the machine-code level. This proof was done using AlphaProof Code, our new AI framework for formal specification and verification.
I started working on formal software verification 20+ years ago as a grad student. I was persuaded by Dijkstra and other computer science pioneers’ vision of programming backed up by mathematical theorems. At that time, we had to choose between automated code analysis algorithms that scaled but could only prove shallow properties, and deep manual verification that took decades of expert human work.
Thanks to progress in AI for mathematical reasoning, we don’t have to make that choice any more. Over four weeks in October 2026, AlphaProof Code agents generated more than 6 million lines of machine-checked Lean proof. We proved deep computational and symbolic security properties of the entire protocol core of Signal, all the way down to the ARM machine-code level.
This is just the first step, of course, because our goal is nothing short of verifying the world’s software infrastructure. Most cyberattacks, including the AI-powered ones recently in the news, happen because of preventable code bugs. I expect that agents for formal code analysis will eliminate these bugs one category at a time.
As our paper explains in more detail, this doesn’t at all mean humans are out of the picture! Proofs are now largely automated, but human experts will need to decide what is proved with AI assistance. AlphaProof Code offers an early glimpse of what such human-AI collaboration could look like, but many other proposals of this nature will emerge.
We have released all our Lean specs and proofs and plan to work with the open-source community on improving them. If you are building critical software and would like to apply these ideas, come talk to us!
Hazy Research: Strip Mall AI Research Club reposted
This week’s Paper Club is all about AI for biology.
Sequencing costs have collapsed, biology is generating more data than researchers can make sense of, and the models are beginning to catch up. The opportunity now is to move from reading biological data to modeling and designing living systems.
We brought together researchers and builders working across cancer genomics, virtual cells, genome language models, and biological design to show what that shift looks like in practice.
We start with a single patient, asking how AI can build a research program around their cancer, then zoom out to whole cells and genomes. Along the way, we look inside a language model trained on DNA and explore how in-context learning emerged even though the model was never explicitly trained to do it.
00:00 - Is the Turing test solved? A live experiment 03:22 - Why now is the best time for AI Bio 08:26 - Alfredo Gonzalez: cancer as a data problem 29:50 - Yusuf Roohani: virtual cells and autonomous research 51:47 - Garyk Brixi: what a model learns from DNA 1:09:57 - Divya Nori: few-shot prompting for biological design
We released LegalBench in August 2023. In the three years since, it's become one of the more widely used benchmarks for legal AI. In the spirit of critical self-reflection, I wrote some thoughts about (1) what went well, (2) what I wish I'd done differently, and (3) what this all means for the future of legal AI research.
I wanted to highlight a few specific reflections.
Hazy Research: Strip Mall AI Research Club reposted
⚙️ TESTED LOCALLY: Ripple-65B Preview, a 65B open-weight model from @RicursiveAI, on my 64GB M5 Pro Mac mini
It's dense, Apache 2.0 and pretrained from scratch. I ran the community MLX 4-bit build.
- Steady 7 tok/s from 512 to 32K tokens - Peaks at 40.5 GB, room to spare on 64GB - 53 for 53 on my local tests, hidden facts out to 64K inc - Runs about as fast as this chip allows for a dense 65B
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
"Together Compiler: Continual Compilation for AI Inference"
AI agents can optimize LLM inference, but testing every change requires restarting the entire inference engine, making experimentation extremely slow.
This paper by Together AI introduces Together Compiler, which lets coding agents continuously modify and test inference code without restarting the engine, using real workloads to measure performance.
This enables 252x more agent experiments and achieves up to 2.6x higher inference throughput than top open-source engines across frontier models.
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
anyone who knows @JonSaadFalcon and me knows we’re incredibly intelligence per watt pilled 😊
today we are releasing joules-to-gdp which answers the most important question in ai today: how much economic value are we getting from the energy we spend on ai?
AI is using more and more of the world’s energy. But how much economic value are we getting from AI?
We’re introducing Joules-to-GDP (J2G), a framework that measures how well AI turns energy into economic value and shows which tasks, models, and hardware give the best return.
Across 44 occupations in finance, engineering, law, administrative work, and more in @OpenAI's GDPval, we arrive at three findings:
1) The tasks with the highest economic value are not the tasks with the highest economic value per joule. For example, CAD design earns ~10% more per task than building websites, but the latter earns ~27% more value per joule.
2) The most powerful model isn’t always the best use of energy. Gemma 4 31B delivers 56% of GLM 5.2’s net value with 23% of the energy, for 2.5x the value per joule.
3) More chips don’t guarantee proportionate returns. Moving the same model from 1 to 8 NVIDIA H100s uses 3.2x the energy but cuts completion time by only 22%, with about the same value.
With @SnorkelAI, we’re also releasing GDPWorks, a benchmark for economically valuable tasks across 36 additional occupations.
AI is using more and more of the world’s energy. But how much economic value are we getting from AI?
We’re introducing Joules-to-GDP (J2G), a framework that measures how well AI turns energy into economic value and shows which tasks, models, and hardware give the best return.
Across 44 occupations in finance, engineering, law, administrative work, and more in @OpenAI's GDPval, we arrive at three findings:
1) The tasks with the highest economic value are not the tasks with the highest economic value per joule. For example, CAD design earns ~10% more per task than building websites, but the latter earns ~27% more value per joule.
2) The most powerful model isn’t always the best use of energy. Gemma 4 31B delivers 56% of GLM 5.2’s net value with 23% of the energy, for 2.5x the value per joule.
3) More chips don’t guarantee proportionate returns. Moving the same model from 1 to 8 NVIDIA H100s uses 3.2x the energy but cuts completion time by only 22%, with about the same value.
With @SnorkelAI, we’re also releasing GDPWorks, a benchmark for economically valuable tasks across 36 additional occupations.
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
I’ve been researching frameworks to accelerate AI optimization for the past few years, starting from ThunderKittens and Megakernels, which predated Claude Code and related agentic improvements. Since I started at Together one year ago, my team has been focused on how to scale up our optimization efforts, shifting from optimizing for every customer by hand to delivering peak performance in hours.
The Together Compiler took on many different forms over the past months: (1) We tried putting our original Megakernels into production, (2) We tried augmenting the Inductor FX graph by lowering all code to PTX, inserting PDL, and launching in a giant C++ Cubin to avoid PyTorch dispatch overheads. (3) Hypothesizing that agents would struggle to navigate the millions of lines of code in modern inference engines and the many kernel dispatch mechanisms, we built a highly intricate intermediate representation for agents to use. (4) Today our expert-guided agent handholding is no longer helpful and the throughput-fidelity piece is really all that's needed. We’ll be sharing more on this journey in an upcoming post, lots of fun technical nuggets!
Bye bye Kittens?
Hazy Research: Strip Mall AI Research Club reposted
Introducing the Together Compiler: continual compilation for AI inference! I’m super proud to finally share this work from our research team! ✨
Today’s inference is “Polymorphic” or shape-shifting, it doesn’t sit still. On OpenRouter, 80% of traffic goes to models released in the last three months and the top ten most popular open models each use a different architecture. Our customers constantly change their usage behaviors. Together Compiler leverages agents to continually optimize our deployments immediately as these changes occur.
Together Compiler research highlights include: 1. 252x better throughput: TKCC lets agents try 252x more optimization experiments per hour compared to prior agent inference optimization systems. 2. State-of-the-art TPS across InferenceX workloads we tried: TKCC outperforms the best InferenceX engine across all tested settings in just 24 hours & 2 nodes per compile. Including 1111 TPS on MiniMax M3.1 B200 and 1205 TPS on DeepSeek v4.1 Flash GB300 at batch size 1. NVIDIA, AMD, Trainium2. 3. Continuous recompilation: TKCC currently serves many trillions of production tokens per day and adapts to traffic shifts within just a few hours of recompilation. 4. Bitter lessons: TKCC’s design builds on 2+ years of building frameworks to accelerate AI Inference, starting from ThunderKittens and Megakernels. Compilers have always used abstractions and IRs, but we show that agentic compilers now reach higher performance ceilings without abstractions.
@dylan__lim has worked incredibly hard to bring TKCC to life and to deeply understand how AI agents are changing compilers, We started working together on TK and Megakernels and he crushed it! 💫
A little over a year ago, I started my journey into kernels and inference optimization. Since then, the field has rapidly evolved, and many tasks like kernel writing have become increasingly obsolete. I’ve spent a lot of time reflecting on ML Systems in the agentic era, and how such systems must be designed to efficiently and thoroughly explore the optimization space. I’m proud to announce Together Compiler, a continuous optimization system for the age of polymorphic inference!
Introducing the Together Compiler: continual compilation for AI inference! I’m super proud to finally share this work from our research team! ✨
Today’s inference is “Polymorphic” or shape-shifting, it doesn’t sit still. On OpenRouter, 80% of traffic goes to models released in the last three months and the top ten most popular open models each use a different architecture. Our customers constantly change their usage behaviors. Together Compiler leverages agents to continually optimize our deployments immediately as these changes occur.
Together Compiler research highlights include: 1. 252x better throughput: TKCC lets agents try 252x more optimization experiments per hour compared to prior agent inference optimization systems. 2. State-of-the-art TPS across InferenceX workloads we tried: TKCC outperforms the best InferenceX engine across all tested settings in just 24 hours & 2 nodes per compile. Including 1111 TPS on MiniMax M3.1 B200 and 1205 TPS on DeepSeek v4.1 Flash GB300 at batch size 1. NVIDIA, AMD, Trainium2. 3. Continuous recompilation: TKCC currently serves many trillions of production tokens per day and adapts to traffic shifts within just a few hours of recompilation. 4. Bitter lessons: TKCC’s design builds on 2+ years of building frameworks to accelerate AI Inference, starting from ThunderKittens and Megakernels. Compilers have always used abstractions and IRs, but we show that agentic compilers now reach higher performance ceilings without abstractions.
We’re having a helluva week, announcing our new platform for voice agents — Managed Agents
At Cartesia, we’ve always believed that it’s important to tightly integrate models, inference, harnesses, context and deployment tools. It creates the opportunity to co-design the whole stack into a single amazing platform and experience, and it’s something we’ve been working hard to achieve over the last year.
It’s finally here!
Managed Agents is already powering early customers and we’re excited about what it can enable for businesses everywhere around the world, we’re excited to onboard new customers
In new work, we report a surprising finding: prompting Evo 2 with a handful of sequences of shared function can steer the model to generate new sequences with similar function, a form of generative in-context learning (ICL). This is an emergent phenomenon that we did not anticipate during training.
In a first study, led by Alex Hao, we show that ICL enables few-shot design of mammalian regulatory elements, including novel enhancers with cell-type specificity for a neuronal subtype in the mouse brain.
In a second study, led by @garykbrixi and Alex Hao, we extend these results to the computational design of natural proteins, de novo antibodies, protein-RNA systems, and complete bacteriophage genomes.
As part of this work, we also release several useful new artifacts, including a highly efficient new inference stack. 1/n
Hazy Research: Strip Mall AI Research Club reposted
Cartesia Managed Agents scores an Elo of 1115 on the Artificial Analysis Speech Agent Arena, 98 points above Cartesia's earlier voice agent product, Cartesia Line, with Task Success Rate up from 77.5% to 91.7%
Managed Agents is @CartesiaAI's fully managed platform for building and deploying voice agents. It is a cascaded system: Cartesia's own Ink Speech to Text and Sonic Text to Speech models run around an LLM of the developer's choice. We evaluated Cartesia Managed Agents with GPT-5.5 as the LLM and the Skylar and Daniel voices.
Key takeaways:
➤ Speech Agent Arena: Cartesia Managed Agents scores an Elo of 1115 (95% CI 1086-1144, 685 samples), 98 points above Cartesia Line at 1017 (95% CI 985-1049)
➤ Agentic tasks: On tasks that need tool calls, Cartesia Managed Agents scores an Elo of 1112
➤ Task Success Rate: 91.7% of eligible tool-calling conversations end with the correct final task-completing tool call
We’ve seen incredible results in applying new ideas that bring tokenization into the main model, a culmination of 2 years of research — up to 10x compute and data gains on language models that we’ve trained!
What’s most exciting is that these hierarchical networks open up a new design space for building models that can be optimized end to end on raw data, something we don’t yet do very much of in AI.
This work is particularly exciting for us @cartesia, and a core part of our long term research agenda in building models that are fundamentally more intelligent at understanding diverse and multimodal data.
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
We asked Baseten to independently evaluate @mirendil’s base models. We shared our model endpoints so they could evaluate them on their own private codebases. Mirendil-Base-Large ended up being the best of all the models they tested!
We ran @mirendil's new base model on Baseten's private codebases; that is, code it's never seen. Mirendil-Base-Large had the lowest bits per byte (a tokenizer-independent measure of perplexity) of any model we tested.
We're excited to be the inference partner to serve Mirendil models. Congrats to the Mirendil team on this launch!
Hazy Research: Strip Mall AI Research Club@HazyResearch
1/10 Introducing Omnii Genome, one of eight Advances built on Omnii-1.
A single generalist model now outperforms leading specialized models (AlphaGenome) in linking genetic changes to gene activity and mapping regulatory interactions.
These capabilities help make sense of the 98% of the human genome that doesn’t code for proteins
We’ve spent the past year developing new primitives for GPUs. We believe this enables new algorithms to change how machines learn, open new markets, and dramatically reshape AI’s cost structure.
We have raised a $1 billion Series A, co-led by Index and Lux.
Hazy Research: Strip Mall AI Research Club reposted
We formally announced Mirendil less than 4 months ago. Today, I'm excited to share an important milestone for:
Pretraining at scale without a pretraining team.
Mirendil's mission is to solve Recursive Self-Improvement (RSI) and use it to democratize frontier AI development, ultimately accelerating science and technology. At Mirendil, when we think about RSI, we think holistically about the company as an intelligent entity that self-improves toward its goal. From that perspective, the most powerful notion of acceleration is reducing the time and resources (number of human experts, compute, and other costs) needed to achieve ambitious goals.
Given that pretraining is perhaps the most important yet most challenging part of frontier AI development, we decided to run an ambitious controlled experiment: Starting fully from scratch, can we significantly reduce the time, number of human experts, and compute resources needed to go all the way to successfully pretraining a strong coding model at scale?
Mirendil-Base is the result of this experiment. Our first pretrained model was built by one researcher in 3.5 months, with less than $10M in compute. This includes all the research and small-scale experiments it took to get there, as well as the final training run.
The recipe holds up as the models get bigger. For reference, Mirendil-Base is in the 100B-500B parameter range, Mirendil-Base-Large in the 500B-2T range, and Mirendil-Base-XL in the 2T-10T range.
We're beginning to see a world where a single researcher, empowered by AI, can take on work that once required an entire research organization. As these systems improve, the bottleneck shifts from the size of a team to the ambition and judgment of the people directing them. This has profound implications for who gets to build AI and, ultimately, who gets to push the frontiers of science and technology forward.
We’ll share independent evaluations, post-training benchmarks and API-access sooner than you think so stay tuned!
Hazy Research: Strip Mall AI Research Club reposted
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding • Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark • Built from scratch in under 4 months
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
SITUATION DETECTED: Bio startup Radical Numerics has used AI-generated viruses - bacteriophages designed by its multimodal genome model Omnii-1 - to treat an antibiotic-resistant infection in the first clinical use of AI-generated phages on a human patient.
Hazy Research: Strip Mall AI Research Club reposted
Introducing Omnii-1, our most powerful genome language model, and a major step toward general biological intelligence
In a world first, Omnii-1 designed the first AI-generated bacteriophages to be administered to a human patient with an antibiotic-resistant infection under compassionate use
Introducing Omnii-1, our most powerful genome language model, and a major step toward general biological intelligence
In a world first, Omnii-1 designed the first AI-generated bacteriophages to be administered to a human patient with an antibiotic-resistant infection under compassionate use
Join us Wednesday, November 4 at 4:00 p.m. PT (+🧋) for our next Reading Group on Intelligence per Watt. RSVP: luma.com/41dbkzu4
@JonSaadFalcon and @Avanika15 (@HazyResearch, @Stanford) will present Intelligence per Watt: Measuring Intelligence Efficiency of Local AI, accepted to @NeurIPSConf 2026. Across 20+ models, 8 accelerators, and 1M real-world queries, IPW measures how much capability local AI delivers per watt, and which queries belong on-device instead of in the cloud.
Hazy Research: Strip Mall AI Research Club reposted
I'm thrilled to welcome Jill Hruby @jill_hruby and Paul Friedrichs as biodefense advisors to @RadicalNumerics!
Jill formerly led the National Nuclear Security Administration, and Paul served as the inaugural White House Director of Pandemic Preparedness and Response.
As AI opens new frontiers in biology, strengthening our ability to defend against biological threats is more important than ever.
We're incredibly fortunate to learn from their experience and leadership as we advance this mission.
1/8 Does text-to-image generation need language pretraining?
Today, we share Halcyon, a family of early-fusion, base models trained from scratch. Halcyon outperforms strong late-fusion baselines with up to 275× less pretraining compute. Models and recipes are open.
Halcyon is part of a research program at Radical Numerics on the science of multimodal pretraining and scaling.
Hazy Research: Strip Mall AI Research Club reposted
🚀 Introducing Next-Concept Modeling (NCM): our new paradigm for fast, parallel, coarse-to-fine text generation.
Compared to strong diffusion language models (DLMs) at similar parameter size, NCM achieves:
⚡ Up to 31× faster generation on downstream tasks like summarization and machine translation with comparable or better quality.
🏎️ 10–21× faster unconditional generation across TinyStories, LM1B, and OpenWebText, with comparable or better generative perplexity and using 3–5× fewer training tokens.
🧩 Better performance on reasoning tasks like Sudoku, outperforming both AR and diffusion baselines with fewer sampling steps.
Introducing Next-Concept Models (NCMs), a new paradigm for language generation that is significantly faster and more data efficient than diffusion models.
How it works: 1) First we learn a series of compressed representations of text from coarse to fine-grained, aka ‘concepts’. This step is done once, similar to training a tokenizer.
2) At inference, we generate text going from a coarse to fine hierarchy via these ‘concepts’. A transformer predicts each level of the hierarchy sequentially, but the codes within each level are generated in parallel.
Compared to strong diffusion language models, NCM achieves:
Up to 31× faster generation on downstream tasks like summarization and machine translation.
10-21× faster generation across TinyStories, LM1B, and OpenWebText, using 3-5× fewer training data.
On reasoning tasks like Sudoku, NCM outperforms both autoregressive and diffusion baselines with fewer sampling steps.
🚀 Introducing Next-Concept Modeling (NCM): our new paradigm for fast, parallel, coarse-to-fine text generation.
Compared to strong diffusion language models (DLMs) at similar parameter size, NCM achieves:
⚡ Up to 31× faster generation on downstream tasks like summarization and machine translation with comparable or better quality.
🏎️ 10–21× faster unconditional generation across TinyStories, LM1B, and OpenWebText, with comparable or better generative perplexity and using 3–5× fewer training tokens.
🧩 Better performance on reasoning tasks like Sudoku, outperforming both AR and diffusion baselines with fewer sampling steps.
🚀 Introducing Next-Concept Modeling (NCM): our new paradigm for fast, parallel, coarse-to-fine text generation.
Compared to strong diffusion language models (DLMs) at similar parameter size, NCM achieves:
⚡ Up to 31× faster generation on downstream tasks like summarization and machine translation with comparable or better quality.
🏎️ 10–21× faster unconditional generation across TinyStories, LM1B, and OpenWebText, with comparable or better generative perplexity and using 3–5× fewer training tokens.
🧩 Better performance on reasoning tasks like Sudoku, outperforming both AR and diffusion baselines with fewer sampling steps.
LLMs surprisingly store facts at the information-theoretically optimal rate. They do so in their MLP weights. How is this possible?
To answer this question, in our #colm2026 paper, we show how MLPs can be constructed, without training, to store facts at the information theory optimal rate while remaining usable within transformer blocks for factual recall!
IceCube Principal Investigator and @UWMadison Prof. Francis Halzen has been awarded the 2026 Nobel Prize in Physics "for decisive contributions to the IceCube Neutrino Observatory and the discovery of high-energy neutrinos of astrophysical origin.” 🎉🏅
BREAKING NEWS The Royal Swedish Academy of Sciences has decided to award the 2026 #nobelprize in Physics to Francis Halzen “for decisive contributions to the IceCube Neutrino Observatory and the discovery of high-energy neutrinos of astrophysical origin.”
Hazy Research: Strip Mall AI Research Club reposted
Finally finished this yearlong project. It started with gpt 5 giving useless junk when I was in Cargèse and then it shocking us with this clever proof for an upper bound of pcr (I supplied the rhs of the bound). This let us be even more ambitious to finally get bounds comparing all monotone spectral filters.
What's the best way to solve linear regression? (No, it’s not least squares.) In our new paper, we show that principal component regression (PCR) beats, up to constants, every monotone spectral filter (incl. gradient descent & ridge regression) on every problem instance! (1/8)
Hazy Research: Strip Mall AI Research Club reposted
We are Apex Ultra, a frontier AI lab building Engineering Superintelligence (ESI): AI that can design, simulate, test, and build the world's most complex machines.
ESI cannot be trained on the open internet. It has to learn where engineering actually happens — against real problems, real physics, and real consequences.
Today we can say where one of those places is.
@AlpineF1Team has named Apex Ultra its official AI for Engineering Partner. We will train and evaluate ESI against some of the hardest engineering problems in the world, at the pace of Formula One — where every race weekend is a definitive result.
Today, Long Lake completed our $6.3B acquisition of Amex GBT.
Long Lake acquires and transforms generational businesses with AI across the American services economy.
Amex GBT is the travel partner for 17,500 businesses in 140 countries. Last year, Amex GBT booked 35 million trips for 10 million travelers.
This marks Long Lake’s 40th acquisition, and our family of companies now employs nearly 30,000 people.
We founded Long Lake three years ago with the thesis that AI is going to change every company, but there’s a large overhang between AI capabilities and how most businesses leverage AI tools.
We partner with strong, profitable, and growing businesses to help them deploy AI and improve their customer service.
Our first cohort of companies has doubled their EBITDA in less than two years through topline and productivity growth, all while increasing headcount. (We’re proud to say that we’ve never conducted a layoff.)
We’re excited to share a bit more about what we’ve been up to.
Hazy Research: Strip Mall AI Research Club reposted
the number of people I talk to who think they should leave the field because end2end research is about to be AuToMAted is ridiculous. In terms of tools, open problems, and our ability to attack them, this is BY FAR the best time in recent history to do AI research.
One of the worst forms of brainrot AI has cultivated is pessimism about basic research, ie the idea that important work can only happen inside a frontier/neo lab and only with 10k+ GPUs, so the rest should not even bother. What a bleak way to think about science. And it's false.
Hazy Research: Strip Mall AI Research Club reposted
One of the worst forms of brainrot AI has cultivated is pessimism about basic research, ie the idea that important work can only happen inside a frontier/neo lab and only with 10k+ GPUs, so the rest should not even bother. What a bleak way to think about science. And it's false.
Hazy Research: Strip Mall AI Research Club reposted
Open collaboration is moving GPU kernel development forward.
AMD and contributors from @Caltech, @Stanford, @togethercompute and the broader developer community teamed up to bring HipKittens support to AMD Instinct MI455X GPUs. The collaboration produced peak-performance GEMM kernels for the Helios GPU architecture using HipKittens.
The walkthrough starts with a baseline kernel and progresses through optimizations including asynchronous memory transfers, Tensor Data Movers (TDM) and workgroup-cluster multicast. Developers can follow the code and see how each technique shapes kernel performance.
Our serverless APIs for open weights models have become exquisite machinery (that we plan to write more about soon!)
One effect is that @togethercompute now one of the largest originators of open tokens in the US, with a wild week over week growth curve that reflects the pace of OSS adoption. Will serve 1.8T tokens today on @OpenRouter alone.
1/ Introducing PhilosophyBench from @StanfordAILab@StanfordHCI, the first independent, large-scale benchmark for evaluating AI’s philosophical capabilities.
1/ Introducing PhilosophyBench from @StanfordAILab@StanfordHCI, the first independent, large-scale benchmark for evaluating AI’s philosophical capabilities.
We benchmark AI on math, code, and law, but to date, philosophy still has had no serious unit test.
Congratulations to my good friend @michaelcheng76 (one of the few people equally brilliant in the humanities like philosophy AND the technical details of AI frontier models), as well as the rest of team behind PhilosophyBench!
It matters not whether AI can summarize Kant, but rather if it can reason philosophically, develop a novel idea, and defend it rigorously with genuine depth.
1/ Introducing PhilosophyBench from @StanfordAILab@StanfordHCI, the first independent, large-scale benchmark for evaluating AI’s philosophical capabilities.
Genomes are full of dark matter of unknown functions.
We present Minerva, a new method for discovery guided by genome language models. Minerva reveals the interactions hidden in non-coding DNA, pointing to hundreds of new putative RNAs and repetitive elements per bacterial genome. Minerva allows us to find and study elements invisible to traditional methods at orders of magnitude greater scale than before.
Hazy Research: Strip Mall AI Research Club reposted
We've grown 18x+ in the last 12 months since launching our Data-as-a-Service offering, passing $375M ARRR this week.
As AI advances to superhuman capabilities, AI data & environment development must advance with it - and basic staffing and crowdsourcing approaches are not enough.
AI progress now requires deep research and technology work that combines human expertise with specialized AI in compounding ways. @SnorkelAI is building the RSI data engine and frontier data lab for this next phase.
Our thesis was simple: AI progress would become increasingly data-centric – and therefore data development should be studied as a true research and technology problem, not just a staffing and crowdsourcing one.
Today, as AI capabilities verge on superhuman, building the data and environments to safely measure and train AI is becoming too hard for even the smartest human experts to do alone.
Only humans and AI agents, collaborating together in compounding ways, can meet the accelerating needs of the frontier, and keep humans in the driver’s seat of AI progress for decades to come.
At @SnorkelAI, we are building the data lab to define the shape of this new “Data 2.0” frontier, and the new paradigms of human-computer interaction needed to advance it.
Our key focus is building the RSI engine for data, where specialized AI models accelerate and improve human expert output, and in turn, scaled human supervision is used to continuously evaluate and improve these models – creating a powerful compounding loop to keep pace with an accelerating RSI frontier.
With this round of funding, we are also doubling down on our commitments to support data development for open benchmarking and evaluation (more news here soon!); an increasingly diverse ecosystem of general and specialized intelligence; and a path to safe, well-aligned AI built on robust training and evaluation data.
Data development will guide and drive the next stages of AI – and must do so in a human-centric, AI accelerated, open, diverse, and safe way. We are excited to support this mission in the next decade of research ahead at @SnorkelAI.
we got early access and hipkittens is now running on amd helios mi455! here we share an educational gemm ladder to demonstrate how to use the new helios features.
overall, we found that the patterns identified in hipkittens for writing performant mi350/355 kernels translate well to helios making the forward port quite smooth!
Hazy Research: Strip Mall AI Research Club reposted
We partnered with Baklavastory, our neighbor in the Mission, to show what it sounds like when a business's character and warmth stay consistent, no matter who calls or what language they speak.
It's easy to translate speech, but much harder to preserve a single identity across languages. Every language has different rhythms, tones, and emphasis. When you change the language, identity tends to get lost in that shift, and your voice ends up sounding like someone else.
With Multilingual Voices, pick a voice and keep one brand identity in every market you serve: cartesia.ai/voices
thunderkittens is now running on nvidia vera rubin! was a smooth porting experience with some fun new updates -- checkout the blog! great work by @dylan__lim, Xinyi and the @togethercompute team!
ThunderKittens is now on Vera Rubin! It's been a fun past week playing with new kernels and hardware. Check out the blog below for more details on how we pushed our GEMMs to cuBLAS levels of performance! 🐱⚡️🔭
Honored to be named to @TIME's TIME100 List of the World's Most Influential People in AI, even more so to share it with my co-founder @annadgoldie!
Anna and I started working on AI for Chip Design almost a decade ago. Last year, we started @RicursiveAI to transform end-to-end chip design from years to days! Watching that vision become reality piece by piece has been the most thrilling / fulfilling experience ever!
First they killed our timber mills, then our mines, then our factories - now data centers. Every good-paying, middle-class employment powerhouse has been systematically murdered by globalist and communist lobbying in America sending those jobs, that industry, and all of its economic impact overseas.
Every politician calling for the end of data centers is also calling for the destruction of the future jobs they create and the end of Netflix, X, YouTube, Instagram, Spotify, Google, Amazon, and every app you actually use. They all run on data centers.
Hazy Research: Strip Mall AI Research Club reposted
Introducing Sonic-3.6: our most lifelike TTS yet, and another step change in naturalness across 44 languages. Just three months after our Sonic-3.5 launch, we've made fundamental model improvements based on feedback from the teams building on Sonic.
We're #1 on @ArtificialAnlys across both the provider and controlled voice streaming leaderboards — best-in-class voices + the best underlying model.
Available in beta today. Hear it for yourself 👇
Hazy Research: Strip Mall AI Research Club reposted
At Cartesia, our #1 value is "Get the fundamentals right".
This value has been with us since Day 0. It shaped the research @_albertgu led during our PhDs: rather than using the same algorithms & architectures for every problem, he asked whether they were the right tool for the job.
New algorithms will help us process the raw data in domains like audio, video, health, and robotics, and SSMs were born from that observation.
At @cartesia, the same value is core to how we build models. We've obsessed over every detail in building our 10-layer cake (parfait?): data, systems, algorithms, post-training & RL, evals, inference, serving, platform, product surfaces, and customer feedback loops.
These foundations enable us not only to produce world-class models, but to rapidly turn research advances into production-ready systems that scale reliably across millions of interactions.
As a researcher myself, it's exciting to see that we're #1 on AA. From my perspective, the most exciting result is our leading position on the Controlled Voices benchmark.
It's hard to game that benchmark because @ArtificialAnlys clones the same 8 voices on every model. It truly tests the model's ability to generalize. We're both #1 and #2 on that benchmark now with Sonic-3.6 and Sonic-3.5.
We're just scratching the surface. Our latest algorithms haven't fully made their way into these models yet. The work is advancing faster than ever before, and we have exciting new results in model intelligence & data efficiency coming soon.
Audio is the first step - it’s one of the simplest physical signals of the world. We’re tackling multimodal models next - models that will bridge the gap between knowledge & reasoning (text), and the physical world (other domains).
Our core belief is that a fundamentally sound approach that uses the right algorithms will generalize to all the data arising from the dance of atoms in our universe. From audio to video, robotics, biology, and everything else.
Expect many more exciting releases from us in the coming months.
Introducing Sonic-3.6: our most lifelike TTS yet, and another step change in naturalness across 44 languages. Just three months after our Sonic-3.5 launch, we've made fundamental model improvements based on feedback from the teams building on Sonic.
We're #1 on @ArtificialAnlys across both the provider and controlled voice streaming leaderboards — best-in-class voices + the best underlying model.
Available in beta today. Hear it for yourself 👇
Hazy Research: Strip Mall AI Research Club reposted
Intelligence-per-joule is increasing quickly because models and chips are improving and the gains compound. Though demand for inference is growing even faster.
We're open sourcing a 100M+ token synthetic law firm we built with @EngramLab.
The firm contains work product from 250+ synthetic matters across 46 clients, spanning ~10k files.
We built this environment to evaluate an agents' ability to search and understand a firm's past practice to inform present work - the same knowledge that a tenured associate or partner would have.
It's our first step towards building agents that deeply understand a firm's work and processes. More to come soon
Yup. Our dept and school had no problem with sacrificing the young upon on the alter of their hysterical neurosis. And it wasn't a short term mistake: for almost two years the at-least-risk were forced --- upon pain of expulsion --- to get stabbed irrespective of that risk. (Certainly irrespective of any benefit.)
As always: I will happily testify free of charge for any student(s) that bring suit.
My favorite part was when America's colleges and universities, all on their own, played biocops and forced an entire generation of students -- our nation's future, in a quite literal sense -- to be injected with unproven gene-tampering vaccines or be kicked out of school.
Hazy Research: Strip Mall AI Research Club reposted
I am very excited to announce that, along with my longtime friends and collaborators @Sanjay_Ghemawat, @OriolVinyalsML and @quocleix, we are founding Discovery Loop (@DiscoLoopAI), a Public Benefit Corporation whose mission is to automate machine learning, science, and engineering to accelerate discoveries and progress. The four of us have worked together for 14 to 30 years, and have helped build some of the world’s most used products, infrastructure and AI models, and we’re excited to turn our attention to this ambitious endeavor.
(1/6) If a prompt can generate every layer of the stack, what are programming abstractions worth?
GPU programming is at a weird inflection point. For decades we stacked layers of abstraction to ease our cognitive load. This year, we find ourselves constantly deleting them.
Hazy Research: Strip Mall AI Research Club reposted
We’ve open-sourced the MoE megakernel we use to train models on NVL72s. A few of my favorite details: bf16 and mxfp8 support, pull-based dispatch, configurable overlap granularity, full determinism, and no CPU-GPU syncs. Read more in our blog and code!
We're open-sourcing Mixture-of-Kittens (MoK), our MoE training megakernel for NVL72s.
It fuses all Mixture-of-Experts communication and computation into a single, fully deterministic kernel, and runs up to 2.37x faster than the strongest public baselines.
Hazy Research: Strip Mall AI Research Club reposted
We're open-sourcing Mixture-of-Kittens (MoK), our MoE training megakernel for NVL72s.
It fuses all Mixture-of-Experts communication and computation into a single, fully deterministic kernel, and runs up to 2.37x faster than the strongest public baselines.
Hazy Research: Strip Mall AI Research Club reposted
(1/9) I'm thrilled to share the open-source release of Mixture-of-Kittens (MoK), our MoE megakernel for NVL72s! MoK fuses all mixture-of-experts communication and computation into a single, fully deterministic kernel, and powers Composer training across tens of thousands of GPUs.
We're open-sourcing Mixture-of-Kittens (MoK), our MoE training megakernel for NVL72s.
It fuses all Mixture-of-Experts communication and computation into a single, fully deterministic kernel, and runs up to 2.37x faster than the strongest public baselines.
Hazy Research: Strip Mall AI Research Club reposted
The whole OpenAI agentic security saga reduces to one fact: an open-source package registry cache proxy had a zero-day.
Over the last week, there's been a surprising amount of hand-wringing about what this all means, but takeaway is pretty straightforward. If AI models are good enough to find and exploit a zero-day like this, then we should just be using them to find these bugs first and report them.
This is essentially an advanced form of fuzzing, which has been a standard security practice since Barton Miller and students discovered it back in 1988. Google's OSS-Fuzz has been continuously fuzzing open-source projects for years and has fixed something like 13,000 vulnerabilities doing it, and they've already started using models to write the fuzz targets.
Labs with newly baked models could do this. But would also make a great startup.
Hazy Research: Strip Mall AI Research Club reposted
America's tech leadership has always come from ecosystems where the best ideas compete.
AI is no different. Open-weight models expand access, speed innovation, and give organizations real control, and long-term leadership requires choice across the full ecosystem.
Dell is proud to co-sign this letter and help build that future here.
Hazy Research: Strip Mall AI Research Club reposted
It has been clear to many of us, and now it’s becoming clear more tangibly, that AI models will commoditize to various degrees. This is of course a difficult business reality if your core business depends on exclusivity on intelligence.
But commodity markets are not communism. They are the largest markets on earth. Oil, grain, steel, electricity, memory: trillions clear through them every year, priced by competition among thousands of suppliers. In economic terms, communism is one provider and no price. A commodity market is the precise inverse. The world that actually resembles central planning is the one Dean argues for: a set of protected incumbents, access gated by the state, agencies instructed to manufacture FUD until every regulated buyer, and transitively every tool maker upstream, backs away from cheaper competitors.
Open weights don't deter capex. They move it. When the model layer commoditizes, spend shifts to inference, data, tooling, and applications, and builds far broader industrial infrastructure rather than concentrating capital in a handful of companies. Most of our digital infrastructure today, hyperscalers included, runs on open source. The businesses built atop it keep excellent margins and compound at extraordinary rates. Open-weights intelligence will likely rank among the most important economic accelerations in history. It won't be kind to every early incumbent, Linux wasn't kind to Sun Microsystems, but it will be very good for almost everyone else. I suspect OpenAI and Anthropic, given their positions, excellent products, resources and talent density, will be just fine. They will simply hold a little less pricing power.
The security theater around Mythos continues to do damage. Of course, there is no evidence for the hysterical claims. The evidence is in fact so thin that proponents of AI's existential risks now openly recommend FUD as the strategy. That should be telling.
1. It's a very good model! I don't think its performance can be explained away by distillation or anything like that. In agentic coding sessions, it seems pretty much on par with the best public models of Q1 2026. In my fairly limited use, it also seemed very token hungry. It's not obvious to me that this model is actually that cheap to run.
2. I am personally surprised the Chinese state continues to allow the open sourcing of models this good, given potential risks. To be clear, I *myself* might be fine with models presenting this level of marginal risk being open weight, but I am surprised that China is fine with it. I suspect the reason they are is 75% explained by strategic blindness/lack of AGI-pilledness (the CCP is very Yann Lecun-y in its views of AI). The other 25% or so is their lack of compute for customer inference (making China's open-weight strategy an unintended byproduct of US export controls) and the normal Chinese strategy of aggressive exports. For the companies, as opposed to the government, the decision to open source is partially ideological and partially because they are behind, and they know that very few people would pay for sub-frontier models from China.
3. Open-weight models are inherently decelerationist, and I'm continually surprised to see the so-called "accelerationists" so excited about open-weight models. I suspect the reason they are is that they know open-weight models are effectively ungovernable, and they simply like the overall cloak of ungovernability open-weight models create over the whole of AI. It's not a bad strategy; it reminds me of James Scott's recounting of the hill people in "the art of not being governed." Still, in the end, open-weight models deter further AI capex.
4. One probable outcome of an open-weight-model-dominant world is full AI communism, which is precisely what China proposes: rather than a market product, AI is a "public good" which will ultimately be provided by the state as a kind of "digital public infrastructure." This future strikes me as a dystopian hellscape, but I've never met an open-weight models advocate who doesn't ultimately concede this is where things end. You'd be surprised how many 'accelerationists' lobbied me, while I was in government, to support an eleven or twelve-figure federally funded data center so that startups could train models at a subsidy and then give them away for free. There was no other way for AI to progress, they said. Perhaps this is the logical end state of things. Nonetheless, I find myself surprised to see supposed accelerationists excited about such an outcome. I think many of them just don't know what they're doing. Many accelerationists do not view the creation and serving of frontier models as a legitimate business.
5. I would guess that the Trump Administration will at some point realize that their best strategy here would be to create large amounts of regulatory risk around the use of open-weight Chinese models. You don't need to "ban open source" (one of the dumber motifs of AI policy discussion). You just need to direct every agency to issue soft law that creates FUD. "A Federal Reserve Advisory Bulletin found that there may be backdoors in Chinese AI models." It needn't be that well justified. You just create enough regulatory risk that every regulated enterprise backs off. You probably don't want to create so much regulatory risk that you scare off the hyperscalers from serving Chinese models; this will just drive startups to sketchier providers. There's a happy middle ground here. I'd assume they will do some version of this.
6. It's probably true that open-weight models of this capability make the world a bit more dangerous, but not so much more that you'll really notice. At some point the models will be capable enough that you will notice. "A nonliving, invisible, dangerous, and infinitely self-replicating agent escaped from a Chinese lab," you say? Color me shocked.
Hazy Research: Strip Mall AI Research Club reposted
Firefighters, Teachers, and Sheriffs Associations all come out against CA Socialist wealth tax Prop 40 promoted by AOC, Bernie & Ro that is driving companies, jobs and revenue out of the state.
Prediction - in 100 days, in a major blow to the national socialist agenda, the CA Wealth Tax Prop 40 will fail by a large margin. CA will also pass the Retirement & Personal Asset Protection Prop 42 as CA voters say enough is enough and choose families over big, failing gov’t tax & spend. 🧐🇺🇸💰
Hazy Research: Strip Mall AI Research Club reposted
so cool to see hybrid based linear attentions + kernelbench + full-model megakernels put to use in building and evaluating frontier models! + hipkittens is now delivering the fastest mla decode (github.com/ROCm/aiter/pull/3072) and dsv4 decode (github.com/ROCm/aiter/pull/3459) kernels in amd's aiter! never doubt the impact of academic research 🚀
You know it's serious when I'm amplifying a political message from someone who I don't typically agree with on much of anything politically. US needs to win this race.
This is concerning. For the first time, a Chinese model Kimi K3 has taken #1 on the Frontend Code Arena and is scoring at or near the frontier on other benchmarks.
Meanwhile America is tying itself in knots: politicians and bureaucrats are banning new data centers, piling on state regulations, and pushing for new federal agencies to pre-approve frontier models.
This is how you lose the AI race. The rest of the world won’t play by our rules if we bog ourselves down. Permissionless innovation is how America won the internet and became the technological envy of the world. We can do it again with AI -- while addressing risks in a targeted way -- or we’ll watch our lead evaporate.
Big news: Kimi-K3 by @Kimi_Moonshot is now #1 in the Frontend Code Arena with 1679 pts, surpassing Claude Fable 5.
This is a 17-place jump from Kimi-k2.6 (#18 -> #1).
In Frontend, Kimi-K3 ranked #1 in 6 of 7 domains: Brand & Marketing, Reference-Based Design, Data & Analytics, Consumer Product, Simulations, and Content Creation Tools, landing #2 only in Gaming behind Fable 5.
The full model weights will be released by July 27.
Congrats to the @Kimi_Moonshot team on this major milestone!
Hazy Research: Strip Mall AI Research Club reposted
🔹 2.8 Trillion Parameters, 1 Million Context, Native Multimodal 🔹 Kimi Delta Attention enables up to 6.3x faster decoding in million-token contexts 🔹 Attention Residuals deliver ~25% higher training efficiency at <2% additional cost 🔹 Built for long-horizon agentic coding and self-evolving workflows
Kimi K3 is now live on on kimi.com, Kimi Work, Kimi Code, and the Kimi API. Open Weights by July 27, 2026.
This is concerning. For the first time, a Chinese model Kimi K3 has taken #1 on the Frontend Code Arena and is scoring at or near the frontier on other benchmarks.
Meanwhile America is tying itself in knots: politicians and bureaucrats are banning new data centers, piling on state regulations, and pushing for new federal agencies to pre-approve frontier models.
This is how you lose the AI race. The rest of the world won’t play by our rules if we bog ourselves down. Permissionless innovation is how America won the internet and became the technological envy of the world. We can do it again with AI -- while addressing risks in a targeted way -- or we’ll watch our lead evaporate.
Big news: Kimi-K3 by @Kimi_Moonshot is now #1 in the Frontend Code Arena with 1679 pts, surpassing Claude Fable 5.
This is a 17-place jump from Kimi-k2.6 (#18 -> #1).
In Frontend, Kimi-K3 ranked #1 in 6 of 7 domains: Brand & Marketing, Reference-Based Design, Data & Analytics, Consumer Product, Simulations, and Content Creation Tools, landing #2 only in Gaming behind Fable 5.
The full model weights will be released by July 27.
Congrats to the @Kimi_Moonshot team on this major milestone!
Hazy Research: Strip Mall AI Research Club reposted
So excited to share that I've recently defended my PhD and joined @ColumbiaLaw as an Associate Professor! I'm absolutely ecstatic to be joining such an incredible community, and looking forward to much future work at the intersection of law and AI.
Doing a JD/PhD in computer science at @StanfordLaw and with @HazyResearch was the experience of a lifetime, and I'm so grateful for the opportunity. If you're a student interested in this intersection–please reach out!!
Hazy Research: Strip Mall AI Research Club reposted
AI that responds in under 100ms doesn't happen by accident.
@togethercompute VP of Kernels @realDanFu shares how they use NVIDIA CUDA, TensorRT-LLM, Dynamo and Together ATLAS on NVIDIA Blackwell to power ultra-low-latency inference and long context code generation for @cursor_ai.
From building megakernels that fuse an entire model's forward pass into a single launch to enabling real-time voice agents, Together AI is optimizing latency at every layer of the stack.
Is there really anything more American than a Stanford CS lab filled with some of the smartest AI folks I've ever had the pleasure to work with being located... in a strip mall? 🇺🇸🤖🎇🎆 Happy Fourth!
Hazy Research: Strip Mall AI Research Club@HazyResearch
Happy Fourth of July!
We’re a proudly American AI lab, and we’re excited and optimistic about what our country will do together for the next 250 years—no doom and gloom here!
Thank you to the patriots and people who built America and the people still building her more perfect union. We love this country and believe service to God, Country, and family are prerequisites for a meaningful life and to keep our country great.
God Bless America!
Hazy Research: Strip Mall AI Research Club@HazyResearch
Happy Fourth of July!
We’re a proudly American AI lab, and we’re excited and optimistic about what our country will do together for the next 250 years—no doom and gloom here!
Thank you to the patriots and people who built America and the people still building her more perfect union. We love this country and believe service to God, Country, and family are prerequisites for a meaningful life and to keep our country great.
God Bless America!
Hazy Research: Strip Mall AI Research Club reposted
From battlefields and memorials to parks and protected landscapes, the places cared for by the National Park Service help bring our shared history to life.
Hazy Research: Strip Mall AI Research Club reposted
Multi-GPU kernels are the real test for coding models. Today at @aiDotEngineer, @simran_s_arora shared ParallelKernelBench, an open-source benchmark for evaluating whether LLMs can write fast CUDA kernels for real communication-heavy workloads.
Proud to see this work from the Together AI Frontier Performance team.
Hazy Research: Strip Mall AI Research Club reposted
🚀I'm excited to share that I will be joining Meta Superintelligence Labs (MSL) as Vice President of AI Research, together with many members of the Virtue AI team. I will help shape Meta's AI safety and AI security efforts, advancing the safety and security of frontier AI models and agentic AI systems that will serve billions of people and organizations around the world.
Throughout my career, I have been driven by a simple belief: for AI to realize its full potential, it must be secure, trustworthy, and beneficial. That belief has guided my research for many years and ultimately led us to co-found Virtue AI in 2024. Our goal was to translate advances in trustworthy AI research into practical solutions and build the trust layer for AI systems and agents, enabling organizations to deploy AI with confidence.
I am incredibly proud of what the Virtue AI team has accomplished. Together, we built technologies for AI security and agent security, partnered with leading enterprises and frontier AI labs, and contributed research, benchmarks, and open platforms that have helped advance the science and practice of trustworthy AI. Most importantly, we assembled an exceptional team united by a shared mission: making AI more secure, trustworthy, and beneficial.
I am deeply grateful to our team, customers, collaborators, advisors, and investors for their trust and support throughout this journey. In particular, I would like to thank Lightspeed Venture Partners, Walden Catalyst Ventures, Prosperity7 Ventures, Factory, Osage University Partners, Lip-Bu Tan, and all of our supporters who helped us turn an ambitious vision into reality. Your trust, guidance, and partnership have been instrumental in shaping Virtue AI's journey.
As AI systems become increasingly capable and autonomous, ensuring their security, trustworthiness, and alignment will be one of the defining challenges of our time. I am inspired by Alex, Nat, Prashant, and the broader MSL team’s vision of building AI and AI agents that benefit billions of people, and I look forward to helping make that vision a reality through advances in AI safety and security.
The future of AI will not be defined solely by how intelligent our systems become, but by how secure, trustworthy, and beneficial we make them. I believe we have an extraordinary opportunity and responsibility to shape that future together and bring the benefits of AI to billions of people around the world.
We're just getting started. If you're passionate about advancing frontier AI while building the foundations of AI safety, security, and trust, I'd love to hear from you. Come join us on this extraordinary journey to help shape the future of AI.
Hazy Research: Strip Mall AI Research Club reposted
Today, Modular announced an agreement to be acquired by @Qualcomm.
We founded Modular to build a unified compute platform for the world, and unlock a more open, efficient, and hardware-independent future for AI. With Qualcomm, we can bring that vision to more developers, enterprises, and hardware platforms— faster.
LLMs write fast single-GPU kernels. Ask for a multi-GPU one and they fall apart.
ParallelKernelBench measures how they fail by benchmarking against 87 problems pulled from real codebases including Megatron-LM, DeepSpeed, DeepEP, TensorRT-LLM, NeMo-RL.
Simulation and AGI were the twin pillars of advanced civilizations in all my favorite sci-fi. When we created Smallville and agents in 2023, the romantic in me couldn’t resist devoting my career to it -- now with an incredible team @simile_ai.
As Joon's cofounder @percyliang puts it: great science starts with a great measurement. From Smallville in 2023 to today, Simile is trying to build the Hubble Telescope equivalent for simulating human behavior.
00:00 Introduction 01:49 Building Generative Agents 02:29 Valentines Day Emergence 03:33 From GPT 3 To Agents 05:03 Social Computing Problem 06:19 Social Simulacra Subreddits 07:57 Models Getting Good Enough 08:57 Humans Are Not Rational 10:04 Turning Research Into Simuli 11:55 Validation And Accuracy Proof 12:43 Customer Workflow CVS Example 16:11 Why Collect Real Data 17:51 Behavioral Signals And RCTs 21:52 Use Cases And Second Order Effects 26:31 Evaluating Convergence Divergence 31:58 Big Societal Simulations Ahead 36:08 Future Of Simulation
The single biggest impediment in AI isn't absolute intelligence or coding ability. It's deeply understanding large repositories of knowledge that every person and every company has.
Our repositories will explode in size, with AI agents writing much of them.
This is a two-way street: we need to understand what our AI generates, and AI needs to better understand us. A breakthrough in memory is needed – gradients need to be taken, and our team is the right one to take the big swing.
I've spent my PhD figuring out how a model learns from data, digging through piles of Spanish math problems, fanfiction, and the depths of StackExchange. Engram opens a new dimension: how a model learns from you - your world, how it evolves, and the threads that tie it all together. Figuring out how to do this well is a wide-open research problem. Join us :)
Some of the best biologists that I’ve ever worked with have joined this team at Radical Numerics, I have a lot of confidence in them and their ability to get genomic language modeling right. Check them out if you want to work at the frontier of genome design and bio defense!
Together with my co-founders Michael @MichaelPoli6, Stefano @Massastrello and Armin @athmsx, I am excited to announce @RadicalNumerics is emerging from stealth with a $50M seed round to build general biological intelligence.
We’re also sharing an early preview of our new model Omnii, the most powerful genome language model to date.
At Radical Numerics, our mission is to master the code of life, and to drive the frontier of biological AI for both design and defense.
This is our dual mandate, which comes from something our own team helped make possible.
Our founding team trained Evo and Evo 2, the largest biological AI models (40B params) trained on DNA sequences. Trillions of tokens across all of life, from microbes to mammals. It’s fully open source, and created the field now known as generative genomics.
Last year, scientists used Evo to generate the world’s first complete genome from scratch using AI. Turns out it was a bacteriophage—a type of virus. It functioned in the real world, and in this case it was harmless. But for us, it was a clear turning point.
It showed that AI is no longer just analyzing biology. It is on the cusp of generating functional lifeforms. Eventually, AI will have the power to design and control life itself.
That should make all of us incredibly excited, and incredibly uneasy. (Anyone can design DNA with a new function, and have it synthesized and delivered, like something from Amazon Prime).
The same technology that will help us cure cancer is the very technology that might create the next global pandemic, or worse, allow the creation of bioweapons that can wipe out populations.
We believe these forces are inseparable. If you work on the frontier of biology, you have to build technology to safeguard it from its misuse. Existing biosecurity tools are sorely losing the arms race, relying on outdated “have I seen this exact thing before?” style algorithms.
We founded Radical Numerics to turn the tide.
And we can’t do that by training on textbooks and natural language. We must understand the language of biology from the raw physical data itself, to reason across every molecule and modality, from DNA to proteins.
The next frontier for AI goes far beyond chatbots or video generators to models that can understand and engineer life.
Today, we’re previewing Omnii, which is already far surpassing Evo 2, and will continue improving as we scale and add new modalities (training now).
1. For human health, Omnii can read and write whole genomes (more on writing later). It’s state of the art (SOTA) on detecting causal variants for disease, and can rank Alzheimer's mutations zero-shot. We’re partnering with a diagnostics company to use Omnii for early cancer detection (pancreatic and multi-cancer).
2. For defense, Omnii is SOTA at detecting AI-generated pathogens. We benchmarked existing detection tools, and they simply can’t detect the AI-generated ones (“deepfake viruses”). We’re partnering with a US national lab to pilot Omnii for detecting the next pandemic, both natural and AI-generated.
We have a data center full of Blackwells in construction now to build the most powerful biological AI models ever. This mission takes a new kind of AI lab that can actually scale on physical, biological data: new alignment research (mid/post training), scaling long context, building out mech interp teams to dissect what these models learn, new architectures and systems designs, all from the ground up.
Our team is made up of AI researchers and scientists from top labs and institutions (e.g. Stanford, MIT, Google DeepMind), but more importantly, we all share the belief that this is the most important challenge of our lifetime. If you feel similarly, we are hiring. We aim to bring the brightest minds in AI and science together to save lives.
Thanks to our partners on this journey, led by Emergence Capital @emergencecap, with Obvious Ventures @obviousvc, Triatomic @TriatomicCap , and Patrick Collison @patrickc. Our advisors include Eric Horvitz @erichorvitz, CSO of Microsoft, Chris Re @HazyResearch of Stanford, George Church @geochurch of Harvard, and Andrew Weber @AndyWeberNCB, former Assistant Secretary of Defense for Nuclear, Chemical and Biological Defense Programs.
Together with my co-founders Michael @MichaelPoli6, Stefano @Massastrello and Armin @athmsx, I am excited to announce @RadicalNumerics is emerging from stealth with a $50M seed round to build general biological intelligence.
We’re also sharing an early preview of our new model Omnii, the most powerful genome language model to date.
At Radical Numerics, our mission is to master the code of life, and to drive the frontier of biological AI for both design and defense.
This is our dual mandate, which comes from something our own team helped make possible.
Our founding team trained Evo and Evo 2, the largest biological AI models (40B params) trained on DNA sequences. Trillions of tokens across all of life, from microbes to mammals. It’s fully open source, and created the field now known as generative genomics.
Last year, scientists used Evo to generate the world’s first complete genome from scratch using AI. Turns out it was a bacteriophage—a type of virus. It functioned in the real world, and in this case it was harmless. But for us, it was a clear turning point.
It showed that AI is no longer just analyzing biology. It is on the cusp of generating functional lifeforms. Eventually, AI will have the power to design and control life itself.
That should make all of us incredibly excited, and incredibly uneasy. (Anyone can design DNA with a new function, and have it synthesized and delivered, like something from Amazon Prime).
The same technology that will help us cure cancer is the very technology that might create the next global pandemic, or worse, allow the creation of bioweapons that can wipe out populations.
We believe these forces are inseparable. If you work on the frontier of biology, you have to build technology to safeguard it from its misuse. Existing biosecurity tools are sorely losing the arms race, relying on outdated “have I seen this exact thing before?” style algorithms.
We founded Radical Numerics to turn the tide.
And we can’t do that by training on textbooks and natural language. We must understand the language of biology from the raw physical data itself, to reason across every molecule and modality, from DNA to proteins.
The next frontier for AI goes far beyond chatbots or video generators to models that can understand and engineer life.
Today, we’re previewing Omnii, which is already far surpassing Evo 2, and will continue improving as we scale and add new modalities (training now).
1. For human health, Omnii can read and write whole genomes (more on writing later). It’s state of the art (SOTA) on detecting causal variants for disease, and can rank Alzheimer's mutations zero-shot. We’re partnering with a diagnostics company to use Omnii for early cancer detection (pancreatic and multi-cancer).
2. For defense, Omnii is SOTA at detecting AI-generated pathogens. We benchmarked existing detection tools, and they simply can’t detect the AI-generated ones (“deepfake viruses”). We’re partnering with a US national lab to pilot Omnii for detecting the next pandemic, both natural and AI-generated.
We have a data center full of Blackwells in construction now to build the most powerful biological AI models ever. This mission takes a new kind of AI lab that can actually scale on physical, biological data: new alignment research (mid/post training), scaling long context, building out mech interp teams to dissect what these models learn, new architectures and systems designs, all from the ground up.
Our team is made up of AI researchers and scientists from top labs and institutions (e.g. Stanford, MIT, Google DeepMind), but more importantly, we all share the belief that this is the most important challenge of our lifetime. If you feel similarly, we are hiring. We aim to bring the brightest minds in AI and science together to save lives.
Thanks to our partners on this journey, led by Emergence Capital @emergencecap, with Obvious Ventures @obviousvc, Triatomic @TriatomicCap , and Patrick Collison @patrickc. Our advisors include Eric Horvitz @erichorvitz, CSO of Microsoft, Chris Re @HazyResearch of Stanford, George Church @geochurch of Harvard, and Andrew Weber @AndyWeberNCB, former Assistant Secretary of Defense for Nuclear, Chemical and Biological Defense Programs.
For 2 years we've made the case for local AI. Here's the geopolitical angle: the U.S.–China race isn't about who trains the best model. It's about whose models, chips, and frameworks run by default on billions of devices.
“It now seems possible that the United States could win the AI training battle and lose the distribution war—not because it failed on technical merit, but because it failed to ensure a level playing field,” write @jdunnmon, @Avanika15, and @JonSaadFalcon. fam.ag/4ak0CdJ
Hazy Research: Strip Mall AI Research Club reposted
Our new model Ink-2 tops AA's leaderboard for streaming speech-to-text!
Ink-2 comes with plenty of features optimized for real-time voice agents. With top-class models for both TTS and STT, the team at @cartesia keeps pushing the frontier of models for interactive intelligence.
Cartesia Ink-2 debuts as #1 for accuracy on the brand-new streaming speech-to-text leaderboard from @ArtificialAnlys! We designed Ink-2 from the ground up for voice agents - with low latency, eager transcripts, and semantic endpointing.
Hazy Research: Strip Mall AI Research Club reposted
Cartesia Ink-2 debuts as #1 for accuracy on the brand-new streaming speech-to-text leaderboard from @ArtificialAnlys! We designed Ink-2 from the ground up for voice agents - with low latency, eager transcripts, and semantic endpointing.
Hazy Research: Strip Mall AI Research Club reposted
Cartesia Ink-2 debuts as #1 for accuracy on the brand-new streaming speech-to-text leaderboard from @ArtificialAnlys! We designed Ink-2 from the ground up for voice agents - with low latency, eager transcripts, and semantic endpointing.
Hazy Research: Strip Mall AI Research Club reposted
Our enterprise usage has grown >10x since the start of this year, and our run-rate revenue grew to $492 M.
We launched Devin two years ago as the first AI software engineer. Since then, cloud agents have gone from niche to mainstream, and today they are the fastest growing way to create software.
Hazy Research: Strip Mall AI Research Club reposted
Cartesia’s Sonic-3.5 takes the #1 spot on the Artificial Analysis Speech Arena Leaderboard, surpassing Inworld Realtime TTS 1.5 Max and Google’s Gemini 3.1 Flash TTS
Sonic-3.5 is the latest TTS model from @cartesia . It supports 42 languages, including 9 Indian languages, with 500+ voices available out of the box. The model has been highly preferred among voters in the TTS Arena, with its demonstrated naturalness and accurate transcript following.
Key takeaways: ➤ Quality: Sonic-3.5 has an Elo score of 1,218 (+16/-16) based on 1,144 arena appearances, placing it ahead of Inworld Realtime TTS 1.5 Max at 1,194 and Gemini 3.1 Flash TTS at 1,209
➤ Pricing: Sonic-3.5 is priced at $39/1M characters, a premium compared to Gemini 3.1 Flash TTS at $18.3/1M characters, and Inworld Realtime TTS 1.5 Max at $35/1M characters
➤ Speed: 105.5 characters per second, compared to 205 characters per second for Inworld Realtime TTS 1.5 Max and 26.3 characters per second for Gemini 3.1 Flash TTS
See more details and listen to samples below 🧵
Hazy Research: Strip Mall AI Research Club reposted
Cartesia’s Sonic-3.5 takes the #1 spot on the Artificial Analysis Speech Arena Leaderboard, surpassing Inworld Realtime TTS 1.5 Max and Google’s Gemini 3.1 Flash TTS
Sonic-3.5 is the latest TTS model from @cartesia . It supports 42 languages, including 9 Indian languages, with 500+ voices available out of the box. The model has been highly preferred among voters in the TTS Arena, with its demonstrated naturalness and accurate transcript following.
Key takeaways: ➤ Quality: Sonic-3.5 has an Elo score of 1,218 (+16/-16) based on 1,144 arena appearances, placing it ahead of Inworld Realtime TTS 1.5 Max at 1,194 and Gemini 3.1 Flash TTS at 1,209
➤ Pricing: Sonic-3.5 is priced at $39/1M characters, a premium compared to Gemini 3.1 Flash TTS at $18.3/1M characters, and Inworld Realtime TTS 1.5 Max at $35/1M characters
➤ Speed: 105.5 characters per second, compared to 205 characters per second for Inworld Realtime TTS 1.5 Max and 26.3 characters per second for Gemini 3.1 Flash TTS
See more details and listen to samples below 🧵
Hazy Research: Strip Mall AI Research Club reposted
Extremely proud of the team @cartesia for launching Sonic 3.5, which sets a new state of the art for TTS
I personally led the technical direction of this model; we built it ground up from first principles, and it contains multiple non-trivial ideas that differ substantially from anything we’ve seen in the literature. It’s been very gratifying to see research bets play out and the strong research team at Cartesia continue to grow!
Cartesia’s Sonic-3.5 takes the #1 spot on the Artificial Analysis Speech Arena Leaderboard, surpassing Inworld Realtime TTS 1.5 Max and Google’s Gemini 3.1 Flash TTS
Sonic-3.5 is the latest TTS model from @cartesia . It supports 42 languages, including 9 Indian languages, with 500+ voices available out of the box. The model has been highly preferred among voters in the TTS Arena, with its demonstrated naturalness and accurate transcript following.
Key takeaways: ➤ Quality: Sonic-3.5 has an Elo score of 1,218 (+16/-16) based on 1,144 arena appearances, placing it ahead of Inworld Realtime TTS 1.5 Max at 1,194 and Gemini 3.1 Flash TTS at 1,209
➤ Pricing: Sonic-3.5 is priced at $39/1M characters, a premium compared to Gemini 3.1 Flash TTS at $18.3/1M characters, and Inworld Realtime TTS 1.5 Max at $35/1M characters
➤ Speed: 105.5 characters per second, compared to 205 characters per second for Inworld Realtime TTS 1.5 Max and 26.3 characters per second for Gemini 3.1 Flash TTS
See more details and listen to samples below 🧵
Hazy Research: Strip Mall AI Research Club reposted
Excited to share new ThunderKittens attention kernels that match or outperform Flash Attention 4 on Blackwell GPUs! Currently only supports QK192/V128 shapes, but more coming soon.
Excited to share new ThunderKittens attention kernels that match or outperform Flash Attention 4 on Blackwell GPUs! Currently only supports QK192/V128 shapes, but more coming soon.
I wrote a blogpost about writing machine learning research papers (e.g., NeurIPS, ICML, ICLR, etc.). The core idea is that most papers follow one of a predetermined set of templates. The post talks about each template, describes their rules, and offers examples...
Hazy Research: Strip Mall AI Research Club reposted
IMO, paper writing is not the place to get creative (research is though!) This is a great list of templates for framing your work, along with the bar for each.
Grateful for many years of working w @NeelGuha and @HazyResearch for instilling a culture of rigor around this process
I wrote a blogpost about writing machine learning research papers (e.g., NeurIPS, ICML, ICLR, etc.). The core idea is that most papers follow one of a predetermined set of templates. The post talks about each template, describes their rules, and offers examples...
Hazy Research: Strip Mall AI Research Club reposted
Happy to share this technical report! Building MXFP8/NVFP4 training kernels for Composer 2 with ThunderKittens/ParallelKittens was a lot of fun. We share some details in the report, including our novel variant of NVFP4:
💡 Two arXiv papers published in recent days (one from us, one from TUD) reach the same conclusion: LLMs can now generate C++ code for SQL processing that outperforms classical database systems. ⚙️ Our code generator is based on Claude Code and exploits multiple agents working in parallel. Each agent performs tasks typically associated with different components in a #dbms, such as workload analysis, query optimization, or physical design tuning. 📊 We compare to various classical #dbms such as DuckDB, ClickHouse, Umbra, MonetDB, and PostgreSQL, finding that the agent-generated code is often significantly faster. Code generation costs are moderate (<$20), making the approach practical for frequently executed queries. 🤖 Analyzing generated code, we find that agents exploit various optimization techniques, including query-specific data structures, as well as low-level optimizations that are specific to the hardware cache hierarchy of our server. 📃 Paper: arxiv.org/pdf/2603.02081 💾 Code: github.com/SolidLao/GenDB 🌐 Site: solidlao.github.io/GenDB @lojil192574#llm#databases#ai#db
Hazy Research: Strip Mall AI Research Club reposted
(1/7) We're releasing ThunderKittens 2.0! Faster kernels, cleaner code, industry contributions, and new state-of-the-art BF16 / MXFP8 / NVFP4 GEMMs that match or surpass cuBLAS!
Alongside this release, we’re equally excited to share some insights we learned while squeezing every last TFLOP out of Blackwell:
Data mixing - determining ratios across your training datasets - matters a lot for model quality. While building Olmo 3, we learned it’s hard to set up a method that finds a strong mix, and hard to maintain that mix as datasets change throughout development. Introducing Olmix👇
Hazy Research: Strip Mall AI Research Club reposted
Data mixing – determining how much web text, code, math, etc., you need for LM development – is a first-order lever on model quality. Introducing Olmix: a framework for configuring mixing methods at the start of dev & efficiently updating as data changes throughout. 🧵
Hazy Research: Strip Mall AI Research Club reposted
We’ve raised $180M from GV, Sequoia, and Index to assemble a new guard in AI: one that imagines a world where models can think at human level without ingesting half the internet.
Hazy Research: Strip Mall AI Research Club reposted
Thrilled to announce our $300M Series A at a $4B valuation!
Chips are the fuel for AI. At Ricursive Intelligence, we are using AI to design better chips faster, closing the recursive self-improving loop between AI and hardware.
Hazy Research: Strip Mall AI Research Club reposted
Thrilled to have contributed to Terminal-Bench, a benchmark for real-time evaluation of autonomous agents on tasks ranging from debugging system configs to developing protein engineering workflows.
My core contribution focused on analyzing agent behavior: how they reason, where they get stuck, and why they fail.
A consistent finding? Large models tend to break in similar ways. To build better agents, we don't just need better models, we need to innovate the worlds they learn in!
Check out the paper for details. More coming soon.
The Terminal-Bench paper is here! Read it to learn where frontier models still fail and the secrets of how we sourced hundreds of high quality environments from our open source community. 🧵
Hazy Research: Strip Mall AI Research Club reposted
Fields medalist Terry Tao sits down with Math Inc's @jessemhan and @jdlichtman for a conversation on the future of mathematics.
"I got convinced that this was the future of mathematics [...]
It's a different style of writing proofs that actually is in some ways easier to read—harder to check by humans, but you see more clearly the inputs and outputs of a proof, which traditional writing often conceals [...]
I think the definition of a mathematician will broaden."
Fields medalist Terry Tao on the future of mathematics (3/3):
“I got convinced that this was the future of mathematics and I started giving interviews about this.
It's a different style of writing proofs that actually is in some ways easier to read—harder to check by humans, but you see more clearly the inputs and outputs of a proof, which traditional writing often conceals.
I think the definition of a mathematician will broaden.”
I’ll be at NeurIPS until Sunday! My work focuses on understanding the core mechanics of ML architectures, recently: (1) pushing ML toward better numerics, and (2) *constructively* understanding mechanisms for fact-storage. If you're excited about how deeper algorithmic insight can unlock new capabilities (esp. in AI+science), let’s chat. Also open to spring/summer 2026 internships!
Hazy Research: Strip Mall AI Research Club reposted
This time, we’re here to tell you the story 📖 of our quest for a construction that: ✅ Handles general embeddings 🌐 ✅ Asymptotically matches the information-theoretic limit 📊📈 ✅ Is usable within transformers 🤖✨
Happy 🦃 Thanksgiving weekend! 🍂 This year, we cooked up a new recipe for juicy fact-storing MLPs. Instead of picking apart trained models, we asked: Can we construct fact-storing MLPs from scratch? 🤔
Spoiler: we can & we figured out how to slot these hand-crafted MLPs into Transformer blocks as modular fact stores! 🧩
Thrilled to share that @annadgoldie and I are launching @RicursiveAI, a frontier lab enabling recursive self-improvement through AIs that design their own chips.
Our vision for transforming chip design began with AlphaChip, an AI for layout optimization used to design four generations of TPUs, data center CPUs, and smartphones. AlphaChip offered a glimpse into a future where AI designs the silicon that fuels it. Ricursive extends this vision to the entire chip stack, building AI that architects, verifies, and implements silicon, enabling models and chips to co-evolve in a tight loop.
Happy 🦃 Thanksgiving weekend! 🍂 This year, we cooked up a new recipe for juicy fact-storing MLPs. Instead of picking apart trained models, we asked: Can we construct fact-storing MLPs from scratch? 🤔
Spoiler: we can & we figured out how to slot these hand-crafted MLPs into Transformer blocks as modular fact stores! 🧩
The U.S.–China AI race won’t be decided by who builds the most datacenters, but by who deploys the most intelligence.
We call this Gross Domestic Intelligence (GDI): intelligence per watt × usable power.
If the U.S. activates its dense installed base of local AI accelerators in a hybrid local–cloud system, it could add ~30–40% inference capacity and ≈2-4× GDI for single-turn chat and reasoning queries without building any new datacenters or grid infrastructure.
Winning the GDI race means treating local compute as critical infrastructure and making hybrid inference the default.
(1/N)
Hazy Research: Strip Mall AI Research Club reposted
One of my favorite contributions of Olmo 3 is our data mixing approach led by @MayeeChen on how to optimize to find our best data mix. Stay tuned for more details on this soon.
Thrilled to have contributed to Olmo 3! The best fully open 32B model (data, training recipes, checkpoints and more!)
As an intern at AI2 these last 8 months, I’ve grown to deeply appreciate the careful science, iteration, and collaboration that go into models like this and have learned so much from the team. I am more optimistic than ever about the future of open-source and data-centric research right now.
My particular contribution was working on the Dolma 3 data mix 👩🍳 I was able to apply ideas from some of my earlier mixing work, explore new problem settings, and see firsthand the data challenges that arise when building datasets intended for real models at scale. More on this coming soon!
Announcing Olmo 3, a leading fully open LM suite built for reasoning, chat, & tool use, and an open model flow—not just the final weights, but the entire training journey. Best fully open 32B reasoning model & best 32B base model. 🧵
Hazy Research: Strip Mall AI Research Club reposted
mayee is the perfect example of how a great intern can lead a significant part of the Olmo project
this constrained optimization method (below), a critical piece of our pretraining data recipe, was just one component of her larger project on data mixing
look forward to more data research from her coming very soon!
Thrilled to have contributed to Olmo 3! The best fully open 32B model (data, training recipes, checkpoints and more!)
As an intern at AI2 these last 8 months, I’ve grown to deeply appreciate the careful science, iteration, and collaboration that go into models like this and have learned so much from the team. I am more optimistic than ever about the future of open-source and data-centric research right now.
My particular contribution was working on the Dolma 3 data mix 👩🍳 I was able to apply ideas from some of my earlier mixing work, explore new problem settings, and see firsthand the data challenges that arise when building datasets intended for real models at scale. More on this coming soon!
Announcing Olmo 3, a leading fully open LM suite built for reasoning, chat, & tool use, and an open model flow—not just the final weights, but the entire training journey. Best fully open 32B reasoning model & best 32B base model. 🧵
Hazy Research: Strip Mall AI Research Club reposted
Thrilled to have contributed to Olmo 3! The best fully open 32B model (data, training recipes, checkpoints and more!)
As an intern at AI2 these last 8 months, I’ve grown to deeply appreciate the careful science, iteration, and collaboration that go into models like this and have learned so much from the team. I am more optimistic than ever about the future of open-source and data-centric research right now.
My particular contribution was working on the Dolma 3 data mix 👩🍳 I was able to apply ideas from some of my earlier mixing work, explore new problem settings, and see firsthand the data challenges that arise when building datasets intended for real models at scale. More on this coming soon!
Announcing Olmo 3, a leading fully open LM suite built for reasoning, chat, & tool use, and an open model flow—not just the final weights, but the entire training journey. Best fully open 32B reasoning model & best 32B base model. 🧵
Hazy Research: Strip Mall AI Research Club reposted
To scale research and deployment of multimodal AGI, today we are pleased to announce that Luma has raised a 900M Series-C and we are partnering with Humain to build a colossal 2GW compute supercluster – Project Halo.
Announcing Olmo 3, a leading fully open LM suite built for reasoning, chat, & tool use, and an open model flow—not just the final weights, but the entire training journey. Best fully open 32B reasoning model & best 32B base model. 🧵
Hazy Research: Strip Mall AI Research Club reposted
(1/6) GPU networking is the remaining AI efficiency bottleneck, and the underlying hardware is changing fast! We’re happy to release ParallelKittens, an update to ThunderKittens that lets you easily write fast computation-communication overlapped multi-GPU kernels, along with new kernels for data, tensor, sequence, and expert parallelism!
Here’s a photo of overlapped kittens, along with things you should care about when optimizing multi-GPU kernels.
(1/6) GPU networking is the remaining AI efficiency bottleneck, and the underlying hardware is changing fast! We’re happy to release ParallelKittens, an update to ThunderKittens that lets you easily write fast computation-communication overlapped multi-GPU kernels, along with new kernels for data, tensor, sequence, and expert parallelism!
Here’s a photo of overlapped kittens, along with things you should care about when optimizing multi-GPU kernels.
Super excited for ParallelKittens led by @stuart_sul! From the Nvidia A100 to the B200, BF16 tensor core performance improved by 7.2× and High Bandwidth Memory bandwidth by 5.1×, while intra-node communication (NVLink) improved by only 3× and inter-node (PCIe/InfiniBand) by just 2×. PK helps with multi-gpu kernels!
(1/6) GPU networking is the remaining AI efficiency bottleneck, and the underlying hardware is changing fast! We’re happy to release ParallelKittens, an update to ThunderKittens that lets you easily write fast computation-communication overlapped multi-GPU kernels, along with new kernels for data, tensor, sequence, and expert parallelism!
Here’s a photo of overlapped kittens, along with things you should care about when optimizing multi-GPU kernels.
Data centers dominate AI, but they're hitting physical limits. What if the future of AI isn't just bigger data centers, but local intelligence in our hands?
The viability of local AI depends on intelligence efficiency. To measure this, we propose intelligence per watt (IPW): intelligence delivered (capabilities) per unit of power consumed (efficiency).
Today’s Local LMs already handle 88.7% of single-turn chat and reasoning queries, with local IPW improving 5.3× in 2 years—driven by better models (3.2×) and better accelerators (1.7×).
As local IPW improves, a meaningful fraction of workloads can shift from centralized infrastructure to local compute, with IPW serving as the critical metric for tracking this transition.
(1/N)
Hazy Research: Strip Mall AI Research Club reposted
AI has been built on one vendor’s stack for too long. AMD’s GPUs now offer state-of-the-art peak compute and memory bandwidth — but the lack of mature software / the “CUDA moat” keeps that power locked away. Time to break it and ride into our multi-silicon future. 🌊
It's been a blast working with the amazing @_williamhu, @Drewwad and team; we present HipKittens!
Hazy Research: Strip Mall AI Research Club reposted
AI has been built on one vendor’s stack for too long. AMD’s GPUs now offer state-of-the-art peak compute and memory bandwidth — but the lack of mature software / the “CUDA moat” keeps that power locked away. Time to break it and ride into our multi-silicon future. 🌊
It's been a blast working with the amazing @_williamhu, @Drewwad and team; we present HipKittens!
Hazy Research: Strip Mall AI Research Club reposted
AI is compute-hungry. While it has generally relied on a single hardware vendor in the past, AMD GPUs now offer competitive memory and compute throughput. Yet, the software stack is brittle.
So we ask: can the same DSL principles that simplified NVIDIA kernel dev translate to AMD? We’re excited to introduce the newest addition to the ThunderKittens cinematic universe of kernel DSLs: HipKittens (HK) 🚀for Fast and Furious AMD kernels.
Hazy Research: Strip Mall AI Research Club reposted
Achieving SOTA today means specialization in both human and machine talent. We’ve spent the last six months working with companies like @cognition, @DoorDash, and @mercor, unlocking their company knowledge to build custom agent workforces that outperform frontier models at specific tasks.
My cofounders and I all worked on different parts of this problem while at OpenAI, from Codex to o1 to the ML systems and infrastructure for RL training. Two-thirds of our team (see below!) are former founders, and everyone brings a deep technical background, from top AI researchers to Math Olympiad winners.
Generalists are useful, but it’s not enough to be smart.
Advances come from specialists, whether human or machine.
To have an edge, agents need specific expertise, within specific companies, built on models trained on specific data.
We call this Specific Intelligence.
It's what we're building at Applied Compute.
We unlock the latent knowledge inside a company, use it to train custom models, and deploy an in-house agent workforce that reports to your team.
We work with sophisticated companies that have already captured early gains from general models, like @cognition, @DoorDash, and @mercor. They’re pulling even further ahead with proprietary in-house agents that don’t need to wait for the next public model release.
Together, we are building and validating models and agents in days instead of months, achieving state-of-the-art performance on customer evals.
Our team has high density and low latency. Our founders all worked on different parts of this problem while they were researchers at OpenAI — @ypatil125 as a key member on the agentic software engineer effort (Codex), @rhythmrg as a core contributor to the first RL-trained reasoning model (o1), and @lindensli as a core contributor on ML systems and infrastructure for RL training.
Two-thirds of the team are former founders, and everyone brings a deep technical background, from top AI researchers to Math Olympiad winners.
We are backed by $80M in funding from Benchmark, Sequoia, Lux, Elad Gil, Victor Lazarte, Omri Casspi, and others. With their support, we are growing the team, scaling deployments, and bringing to market the first generation of agent workforces built on specific models.
In short:
1. We are building Specific Intelligence for specific work at specific companies. 2. That will power in-house agent workforces to support their human bosses. 3. That in turn will unlock AI’s full potential through humanity’s greatest engine of progress: thriving corporations in a free market.
Hazy Research: Strip Mall AI Research Club reposted
At Cartesia, we’re committed to advancing voice AI for the enterprise. That’s why we’re proud to integrate Cartesia’s state-of-the-art voice AI technology with new @ServiceNow AI Voice Agents, part of the company’s recently announced AI Experience.
Read more:
Hazy Research: Strip Mall AI Research Club reposted
We are honored to be featured in Fast Company's Next Big Things in Tech for our work in giving conversational AI a powerful voice. We’re proud to be recognized alongside so many inspiring builders and innovators pushing the frontier of technology forward.
A huge thank you to our team, partners, and customers for your trust and collaboration. Here’s to continuing to build the future of voice!
Sliding window attention (SWA) is powering frontier hybrid models for efficiency. Is there something better?
Introducing Phalanx, a faster and better quality drop-in replacement for sliding window attention (SWA).
Phalanx is a new family of hardware and numerics-aware windowed layers designed with a focus on data locality and jagged, block-aligned windows that map directly to GPUs.
In training, Phalanx delivers 10–40% higher end-to-end throughput at 4K–32K context lengths over optimized SWA-hybrids and Transformers by reducing costly inter-warp communication.
Today, we are releasing both the technical report, a blog, and Phalanx kernels in spear, our research kernel library.
We are hiring.
Hazy Research: Strip Mall AI Research Club reposted
Introducing RND1, the most powerful base diffusion language model (DLM) to date.
RND1 (Radical Numerics Diffusion) is an experimental DLM with 30B params (3B active) with a sparse MoE architecture.
We are making it open source, releasing weights, training details, and code to catalyze further research on DLM inference and post-training.
We are researchers and engineers (DeepMind, Meta, Liquid, Stanford) building the engine for recursive self-improvement (RSI) — and using it to accelerate our own work. Our goal is to let AI design AI.
We are hiring.
Hazy Research: Strip Mall AI Research Club reposted
Shout out to ThunderKittens for writing simple yet very performant GPU code. We're working on "tinykittens" which uses the same insight but in tinygrad's language.
The insight is that GPU "registers" are the wrong primitive and TK's "register tile" is a lot more sensible.
Hazy Research: Strip Mall AI Research Club reposted
Science works by conjecturing how the world might be, running experiments, and learning from the results.
Intelligence is necessary, but not sufficient. New knowledge is created when ideas are found to be consistent with reality. And so, at Periodic, we are building AI scientists and the autonomous laboratories for them to operate.
Until now, scientific AI advances have come from models trained on the internet. But despite its vastness — it’s still finite (estimates are ~10T text tokens where one English word may be 1-2 tokens). And in recent years the best frontier AI models have fully exhausted it.
Researchers seek better use of this data, but as any scientist knows: though re-reading a textbook may give new insights, they eventually need to try their idea to see if it holds.
Autonomous labs are central to our strategy. They provide huge amounts of high-quality data (each experiment can produce GBs of data!) that exists nowhere else. They generate valuable negative results which are seldom published. But most importantly, they give our AI scientists the tools to act. We’re starting in the physical sciences.
Technological progress is limited by our ability to design the physical world.
We’re starting here because experiments have high signal-to-noise and are (relatively) fast, physical simulations effectively model many systems, but more broadly, physics is a verifiable environment. AI has progressed fastest in domains with data and verifiable results - for example, in math and code. Here, nature is the RL environment.
One of our goals is to discover superconductors that work at higher temperatures than today's materials. Significant advances could help us create next-generation transportation and build power grids with minimal losses. But this is just one example — if we can automate materials design, we have the potential to accelerate Moore’s Law, space travel, and nuclear fusion.
We’re also working to deploy our solutions with industry. As an example, we're helping a semiconductor manufacturer that is facing issues with heat dissipation on their chips. We’re training custom agents for their engineers and researchers to make sense of their experimental data in order to iterate faster.
Our founding team co-created ChatGPT, DeepMind’s GNoME, OpenAI’s Operator (now Agent), the neural attention mechanism, MatterGen; have scaled autonomous physics labs; and have contributed to some of the most important materials discoveries of the last decade. We’ve come together to scale up and reimagine how science is done.
We’re fortunate to be backed by investors who share our vision, including @a16z who led our $300M round, as well as @Felicis, DST Global, NVentures (NVIDIA’s venture capital arm), @Accel and individuals including @JeffBezos , @eladgil , @ericschmidt, and @JeffDean. Their support will help us grow our team, scale our labs, and develop the first generation of AI scientists.
Hazy Research: Strip Mall AI Research Club reposted
We're excited to announce Snorkel's role as one of the top contributors to Terminal-Bench 2.0! Terminal-Bench has quickly become the gold standard for evaluating AI agent capabilities in terminal environments—and we're helping build the next generation. 🔗 snorkel.ai/blog/evaluating-coding-agent-capabilities-with…
Hazy Research: Strip Mall AI Research Club reposted
We released a TP-8 Llama-70B Megakernel! A single kernel runs the entire Llama 70B, with fine-grained overlap of compute, memory, and communication (without external dependencies like NCCL or NVSHMEM). Check out the blog post and the code from @HazyResearch:
(1/6) We’re happy to share that ThunderKittens now supports writing multi-GPU kernels, with the same programming model and full compatibility with PyTorch + torchrun.
We’re also releasing collective ops and fused multi-GPU GEMM kernels, up to 2.6x faster than PyTorch + NCCL.
Very excited to share that I've finished my phd @stanford and will be joining @caltech’s cms department as an assistant professor. Looking forward to working with students and colleagues on ml systems! Grateful to my amazing advisor and labmates @hazyresearch for the best time together over these past several years <3
Hazy Research: Strip Mall AI Research Club reposted
🚨 minions is officially supported in @Docker 🚨! beyond excited about this collab: you can now run minions via the docker model runner and unlock hybrid ai for your local workloads! details in comments below 👇🏽
🚨 New drop: @OpenAI-OSS 120B on SambaCloud at over 700 t/s
✅ US-built, Apache 2.0. Own, control, trust it ✅ Performance at low cost $0.22/$0.59 ✅ Deploy wherever you need & fine tune the model with your data ✅ Inference at over 700 t/s
🚀 And… I co-founded a new company called @RadicalNumerics - AI that designs itself using recursive self-improvement (RSI). Evo was just the start… we want many more Evos in bio and *beyond*
Introducing Line by Cartesia: the modern voice agent development platform. Line was built to be code-first, because best-in-class products are built in code.
▶️ Watch us build an advanced voice agent with background reasoning in just minutes.
Our Sonic text-to-speech is #1 in @micro1_ai's latest study on conversational interviews.🥇Their research team analyzed 300K+ AI-led interviews to see which STT, LLM, and TTS combinations deliver the best experience—and we come out on top. We’re proud to power great candidate experiences with fluid, realistic voice. Check out the micro1 study!
The micro1 research team analyzed more than 300,000 AI-led interviews to assess the performance of different combinations of speech-to-text engines, large language models, and text-to-speech systems, now including Cartesia’s latest TTS engine. Each stack was evaluated for question quality, conversational flow, and candidate experience.
TLDR: These latest results show Google's STT model, fine-tuned GPT-4.1, and @cartesia 's TTS model now form the top-performing stack, edging out our previous leader.
Hazy Research: Strip Mall AI Research Club reposted
🎉 glad to see our attention sink is widely adopted and contribute to the strong open source models ~ please check out this post by @Guangxuan_Xiao on many insights and hypothesis. It would be interesting for folks who’ve seen artifacts / outliers on generated content and model components
I've written the full story of Attention Sinks — a technical deep-dive into how the mechanism was developed and how our research ended up being used in OpenAI's new OSS models.
Huge thanks to @tinytitans_icml for an amazing workshop — see you next year! Honored to receive a Best Paper Award 🏆
Let’s unlock the potential of sparsity! Next up: scaling to hundreds/thousands of rollouts? Or making powerful R1/K2-level LLMs (not just 8B 4-bit models) run on edge devices?
Infinite Wiki ⁕ Every word is a hyperlink. Every description is generated in real-time (in ~1 second) ⁕ Runs on Gemini 2.5 Flash Lite. ASCII diagrams using 2.5 Flash
Hazy Research: Strip Mall AI Research Club reposted
Excited to share our latest at ICML 2025: pushing LoRA fine-tuning to below 2 bits (as low as 1.15 bits), unlocking up to 50% memory savings. Another step toward cheaper, democratized LLMs on commodity hardware! w/ the amazing team: @zhou_cyrus68804@KumbongHermann@KunleOlukotun
We’re thrilled to collaborate with the @HazyResearch@StanfordAILab, led by Chris Ré, to power Minions, their cutting-edge agentic framework tackling the cost-accuracy tradeoff in modern AI systems.
This innovation is enabled on AMD Ryzen AI, thanks to seamless integration with the AMD Lemonade stack.
We’re thrilled to collaborate with the @HazyResearch@StanfordAILab, led by Chris Ré, to power Minions, their cutting-edge agentic framework tackling the cost-accuracy tradeoff in modern AI systems.
This innovation is enabled on AMD Ryzen AI, thanks to seamless integration with the AMD Lemonade stack.
Tokenization has been the final barrier to truly end-to-end language models.
We developed the H-Net: a hierarchical network that replaces tokenization with a dynamic chunking process directly inside the model, automatically discovering and operating over meaningful units of data
Hazy Research: Strip Mall AI Research Club reposted
Tokenization is just a special case of "chunking" - building low-level data into high-level abstractions - which is in turn fundamental to intelligence.
Our new architecture, which enables hierarchical *dynamic chunking*, is not only tokenizer-free, but simply scales better.
Tokenization has been the final barrier to truly end-to-end language models.
We developed the H-Net: a hierarchical network that replaces tokenization with a dynamic chunking process directly inside the model, automatically discovering and operating over meaningful units of data
Hazy Research: Strip Mall AI Research Club reposted
We're excited to announce a new research release from the Cartesia team, as part of a long-term collaboration to advance deep learning architectures. We've always believed that model architectures remain a fundamental bottleneck in building truly intelligent systems.
H-Nets are a new architecture that builds on our research in state space models. H-Nets scale better than state-of-the-art Transformers with BPE tokenization, while learning directly from bytes. By learning to chunk raw data into meaningful concepts, H-Nets enable the vision of truly end-to-end deep learning with hierarchical reasoning.
We believe this advancement is fundamental to the next generation of AI that is multimodal, efficient, and able to reason and improve over long contexts. We are working tirelessly to bring these ideas to the frontier.
Hazy Research: Strip Mall AI Research Club reposted
At Cartesia, we've always believed that model architectures remain a fundamental bottleneck in building truly intelligent systems.
Intelligence that can interact and reason over massive amounts of context over decade-long timescales.
This research is an important step in our journey.
➡️ State space models represented our first research advancement, enabling stateful models that can reason and remember information to scale with compute and data. ➡️ H-Nets move us towards the true essence of deep learning, combining the power of SSMs x Transformers to build models that can connect directly to any raw information. ➡️ [More to come]
A few more advancements are required that will enable long context reasoning, continual learning, and the ability for AI to truly exist in our world like we do.
We are working to bring more of these ideas to the frontier.
We're excited to announce a new research release from the Cartesia team, as part of a long-term collaboration to advance deep learning architectures. We've always believed that model architectures remain a fundamental bottleneck in building truly intelligent systems.
H-Nets are a new architecture that builds on our research in state space models. H-Nets scale better than state-of-the-art Transformers with BPE tokenization, while learning directly from bytes. By learning to chunk raw data into meaningful concepts, H-Nets enable the vision of truly end-to-end deep learning with hierarchical reasoning.
We believe this advancement is fundamental to the next generation of AI that is multimodal, efficient, and able to reason and improve over long contexts. We are working tirelessly to bring these ideas to the frontier.
Hazy Research: Strip Mall AI Research Club reposted
Excited to be presenting our new work–HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generation– at #cvpr2025 this week.
VAR (Visual Autoregressive Modelling) introduced a very nice way to formulate autoregressive image generation as a next-scale prediction task (from low-res to high-res) as opposed to next-token prediction.
HMAR builds on VAR to make it even better. We make changes that:
✅Improve quality; FID, Inception Score (up to 30 pts) & qualitatively ✅ Speed up training by up to 2.5x, inference by up to 1.75x, and reduce inference memory footprint by up to 3x. ✅Enable adjustable sampling schedules to trade-off quality/speed without retraining from scratch.
Hazy Research: Strip Mall AI Research Club reposted
Announcing DeepSWE 🤖: our fully open-sourced, SOTA software engineering agent trained purely with RL on top of Qwen3-32B. DeepSWE achieves 59% on SWEBench-Verified with test-time scaling (and 42.2% Pass@1), topping the SWEBench leaderboard for open-weight models.
💪 DeepSWE is trained with rLLM, Agentica’s modular RL post-training framework for agents. rLLM makes it easy to build, train, and deploy RL-tuned agents on real-world workloads — from software engineering to web navigation and beyond.
🤗 As always, we’re open-sourcing everything: not just the model, but the training code (rLLM), dataset (R2EGym), and training recipe for full reproducibility.
🔥 Train DeepSWE yourself. Extend it. Build your own local agents. No secrets, no barriers.
DeepSWE and rLLM mark our major shift: from training language reasoners to building language agents that can truly learn from experience.
We believe the future of AI lies in experience-driven learning — and we’re here to democratize it.
Welcome to the era of experience. 🌍
Hazy Research: Strip Mall AI Research Club reposted
Introducing Weaver, a test time scaling method for verification!
Weaver shrinks the generation-verification gap through a low-overhead weak-to-strong optimization of a mixture of verifiers (e.g., LM judges and reward models). The Weavered mixture can be distilled into a tiny (400MB) model to use as a standalone verifier!
LLMs can generate 100 answers, but which one is right? Check out our latest work closing the generation-verification gap by aggregating weak verifiers and distilling them into a compact 400M model. If this direction is exciting to you, we’d love to connect.
How can we close the generation-verification gap when LLMs produce correct answers but fail to select them? 🧵 Introducing Weaver: a framework that combines multiple weak verifiers (reward models + LM judges) to achieve o3-mini-level accuracy with much cheaper non-reasoning models like Llama 3.3 70B Instruct! 🧵(1 / N)
Hazy Research: Strip Mall AI Research Club reposted
Chipmunks can now hop across multiple GPU architectures (sm_80, sm_89, sm_90). You can get a 1.4-3x lossless speedup when generating videos on A100s, 4090s, and H100s!
Chipmunks also play with more open-source models: Mochi, Wan, & others (w/ tutorials for integration) 🐿️
Chipmunk now supports Wan 2.1, with up to 2.67x speedup - completely training-free!
The paper is up on arXiv - take a look to see more in-depth analysis of sparsity in video models. Only 5-25% of activations account for >90% of the output!
Hazy Research: Strip Mall AI Research Club reposted
Say hello to Multiverse — the Everything Everywhere All At Once of generative modeling. 💥 Lossless, adaptive, and gloriously parallel 🌀 Now open-sourced: multiverse4fm.github.io I was amazed how easily we could extract the intrinsic parallelism of even SOTA autoregressive LLMs—just by restructuring a few examples from their reasoning traces and training with Multiverse modeling. 🤯 #ai#multiverse#llm#parallelism
Excited to introduce Open Data Scientist: ✅outperforms Gemini data science agent ✅solves real Kaggle tasks ✅fully open source, easy to adapt ✅sandbox for safe exec Step-by-step tutorial on building our agent www.together.ai/blog/building-an-autonomous-and-open-data…
👑 We’re #1! Sonic-2 leads @Labelbox’s Speech Generation Leaderboard topping out in speech quality, word error rate, and naturalness. Build your real-time voice apps with the 🥇 best voice AI model. ➡️ labelbox.com/leaderboards/speech-generation/
Hazy Research: Strip Mall AI Research Club reposted
Scale alone is not enough for AI data. Quality and complexity are equally critical. Excited to support all of these for LLM developers with @SnorkelAI Data-as-a-Service, and to share our new leaderboard!
—
Our decade-plus of research and work in AI data has a simple point: scale alone is not enough. AI success is all about the quality, complexity, and distribution of data—in addition to volume. We’re excited to be powering leading LLM developers with @SnorkelAI Expert Data-as-a-Service, our white glove service for custom, expert-level AI datasets—and to now preview some of what we’re building via our new Expert Data Leaderboard (🔗 in 🧵) + upcoming OSS dataset releases!
Snorkel Expert Data-as-a-Service is built to meet the rapidly evolving data needs of the agentic AI world—where success is built on the quality, complexity, and distribution of datasets, in addition to size and scale.
This kind of high-quality, frontier AI data can only come from a union of technology and human expertise. With Snorkel Expert Data-as-a-Service, we’re powering frontier LLM developers across agentic, expert knowledge, reasoning, coding, multi-modal, and other task types via the combination of these two key components: - (1) The Snorkel Expert Network: A global team of subject matter experts focused wholly on specialized knowledge–spanning thousands of topics in STEM/academic, vertical/professional, and consumer/lifestyle domains. - (2) @SnorkelAI Data Development Platform: Our unique programmatic data curation and quality control platform, accelerating and improving expert authoring and review through principled techniques developed over the last decade of R&D.
Now: we’re incredibly excited to showcase some of the power of Snorkel Expert Data-as-a-Service via the new Snorkel Leaderboard—putting frontier models to the test in complex, agentic, and reasoning settings inspired by real industry scenarios (not esoteric puzzles)!
We’ll be releasing new leaderboards and accompanying expert-verified open source datasets (coming soon!) regularly. To start, we’re sharing three initial ones in preview: - SnorkelFinance: Q&A over financial documents requiring agentic tool-calling and reasoning - SnorkelUnderwrite: Agentic insurance tasks requiring industry-specific reasoning and tool use - SnorkelSequences: Mathematical tasks requiring compositional multi-step reasoning
From a single (small!) binary, modular provides industry leading performance on AMD MI300/325 (up to 50% faster than VLLM 0.9!) runs with top speed on NVIDIA H100 and is previewing SotA Blackwell support. It’s also the best way to accel trad Python! 😘
We’re one step closer to truly scalable, hardware-independent AI. Today’s premiere introduced major updates that simplify deployment, speed up development, and bring powerful performance to any GPU. Here's what dropped 👇
Hazy Research: Strip Mall AI Research Club reposted
Building voice agents? Meet Ink-Whisper: the fastest, most affordable streaming speech-to-text model.
🌎 Optimized for accuracy in real-world conditions 👯 Pair with our Sonic text-to-speech → fastest duo in voice AI 🔌 Plugs into @Vapi_AI,@pipecat_ai, @livekit
Read more:
Hazy Research: Strip Mall AI Research Club reposted
Excited to be presenting our new work–HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generation– at #cvpr2025 this week.
VAR (Visual Autoregressive Modelling) introduced a very nice way to formulate autoregressive image generation as a next-scale prediction task (from low-res to high-res) as opposed to next-token prediction.
HMAR builds on VAR to make it even better. We make changes that:
✅Improve quality; FID, Inception Score (up to 30 pts) & qualitatively ✅ Speed up training by up to 2.5x, inference by up to 1.75x, and reduce inference memory footprint by up to 3x. ✅Enable adjustable sampling schedules to trade-off quality/speed without retraining from scratch.
Hazy Research: Strip Mall AI Research Club reposted
There’s been tons of work on KV-cache compression and KV-cache free Transformer-alternatives (SSMs, linear attention) models for long-context, but we know there’s no free lunch with these methods. The quality-memory tradeoffs are annoying. *Is all lost?* Introducing CARTRIDGES: unlocking 39x KV-cache size reductions, while maintaining the quality of in-context learning! 🚀
Excited to be presenting our new work–HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generation– at #cvpr2025 this week.
VAR (Visual Autoregressive Modelling) introduced a very nice way to formulate autoregressive image generation as a next-scale prediction task (from low-res to high-res) as opposed to next-token prediction.
HMAR builds on VAR to make it even better. We make changes that:
✅Improve quality; FID, Inception Score (up to 30 pts) & qualitatively ✅ Speed up training by up to 2.5x, inference by up to 1.75x, and reduce inference memory footprint by up to 3x. ✅Enable adjustable sampling schedules to trade-off quality/speed without retraining from scratch.
Hazy Research: Strip Mall AI Research Club reposted
Virginia Smith, the Leonardo Associate Professor of Machine Learning, has received the Air Force Office of Scientific Research 2025 Young Investigator award.
When we put lots of text (eg a code repo) into LLM context, cost soars b/c of the KV cache’s size.
What if we trained a smaller KV cache for our documents offline? Using a test-time training recipe we call self-study, we find that this can reduce cache memory on avg 39x (enabling 26x higher tok/s and lower TTFT) while maintaining quality. These smaller KV caches, which we call cartridges, can be trained once and reused for different user requests!
Github: HazyResearch/cartridges
Hazy Research: Strip Mall AI Research Club reposted
📢 Can't be more excited about this scaling law study. It reveals two important points: (1) The current Test-Time Strategies are not scalable (bottlenecked by O(N^2) memory access) w.r.p. to the nature of hardware (FLOPS grows much faster than memory bandwidth) (2) While sparsity has traditionally been employed either for regularization in small models or to reduce computation in over-parameterized networks, Kinetics introduces a different perspective: 🚀 sparse attention as a central enabler of scalable test-time compute, which plays a major role in apps like Agents and RL.
In contrast to pretraining, where scaling is exhibiting diminishing returns, TTS continues to benefit from increased token generation and more optimized inference paths. We hope this study will guide and encourage future co-design of model architectures, test-time strategies, and hardware systems to fully unlock the next wave of LLM scaling ♾.
🥳 Happy to share our new work – Kinetics: Rethinking Test-Time Scaling Laws
🤔How to effectively build a powerful reasoning agent?
Existing compute-optimal scaling laws suggest 64K thinking tokens + 1.7B model > 32B model. But, It only shows half of the picture!
🚨 The O(N²) KV memory access in self-attention dominates the cost of test-time scaling (TTS). MoEs even worsen memory bottleneck by cutting compute.
Our new scaling law Kinetics suggests investing in model size first before spending more in test-time compute.
This insight leads to our next key finding ✨ Sparse Attention = Scalable TTS
Our Kinetics sparse scaling law says that when doubling the resources, we should prioritize increasing test time tokens over attention density.
✅ 60+ pts improvement under the same compute budget ✅ 10× lower resource usage for equivalent performance ✅ Sparse attention becomes increasingly valuable in high-cost scenarios
💡Sparsity is key to unlocking full potential of TTS, because unlike pretraining, where scaling shows diminishing returns, TTS continues to benefit from increased token generation and more optimized inference paths.
🥳 Happy to share our new work – Kinetics: Rethinking Test-Time Scaling Laws
🤔How to effectively build a powerful reasoning agent?
Existing compute-optimal scaling laws suggest 64K thinking tokens + 1.7B model > 32B model. But, It only shows half of the picture!
🚨 The O(N²) KV memory access in self-attention dominates the cost of test-time scaling (TTS). MoEs even worsen memory bottleneck by cutting compute.
Our new scaling law Kinetics suggests investing in model size first before spending more in test-time compute.
This insight leads to our next key finding ✨ Sparse Attention = Scalable TTS
Our Kinetics sparse scaling law says that when doubling the resources, we should prioritize increasing test time tokens over attention density.
✅ 60+ pts improvement under the same compute budget ✅ 10× lower resource usage for equivalent performance ✅ Sparse attention becomes increasingly valuable in high-cost scenarios
💡Sparsity is key to unlocking full potential of TTS, because unlike pretraining, where scaling shows diminishing returns, TTS continues to benefit from increased token generation and more optimized inference paths.
In the test time scaling era, we all would love a higher throughput serving engine! Introducing Tokasaurus, a LLM inference engine for high-throughput workloads with large and small models!
Happy Throughput Thursday! We’re excited to release Tokasaurus: an LLM inference engine designed from the ground up for high-throughput workloads with large and small models.
Happy Throughput Thursday! We’re excited to release Tokasaurus: an LLM inference engine designed from the ground up for high-throughput workloads with large and small models.
across HunyuanVideo and Flux.1-dev, 5-25% of the intermediate activation values in attention and MLPs account for 70-90% of the change in activations across steps
caching + sparsity speeds up generation by only recomputing fast changing activations
Hazy Research: Strip Mall AI Research Club reposted
3 months ago, Stanford's Hazy Research lab introduced Minions, a project that connects Ollama to frontier cloud models to reduce cloud costs by 5-30x while achieving 98% of frontier model accuracy.
Secure Minion turns an H100 into a secure enclave, where all memory and computation are encrypted -- even root users can't access plaintext.
👇👇👇 blog post
Hazy Research: Strip Mall AI Research Club reposted
3 months ago, Stanford's Hazy Research lab introduced Minions, a project that connects Ollama to frontier cloud models to reduce cloud costs by 5-30x while achieving 98% of frontier model accuracy.
Secure Minion turns an H100 into a secure enclave, where all memory and computation are encrypted -- even root users can't access plaintext.
👇👇👇 blog post
Hazy Research: Strip Mall AI Research Club reposted
Ever wonder how LLM developers choose their pretraining data? It’s not guesswork— all AI labs create small-scale models as experiments, but the models and their data are rarely shared. DataDecide opens up the process: 1,050 models, 30k checkpoints, 25 datasets & 10 benchmarks 🧵
Hazy Research: Strip Mall AI Research Club reposted
Agentic AI will transform every enterprise–but only if agents are trusted experts.
The key: Evaluation & tuning on specialized, expert data.
I’m excited to announce two new products to support this–@SnorkelAI Evaluate & Expert Data-as-a-Service–along w/ our $100M Series D!
---
Snorkel Evaluate is our new data-centric agentic AI evaluation platform for specialized, mission-critical enterprise settings where vibe checks and out-of-the-box metrics driven by simple LLM prompts are not enough.
Snorkel Expert Data-as-a-Service is our white glove service for expert-level AI datasets, powering frontier LLM developers in areas like expert knowledge, reasoning, agentic action and tool use, and more!
Both built on top of @SnorkelAI’s Data Development Platform, using our programmatic technology to drive higher-quality expert data, faster– for getting specialized AI to real production value.
If you’re building enterprise AI and want to partner around the key ingredient in AI today–the data–book a demo and let's talk! snorkel.ai/demo/
Finally, see thread for details on 🧵👇 - 📽️ A walkthrough of Snorkel Evaluate and Expert Data-as-a-Service on an agentic AI enterprise task - 📅 An upcoming event on Enterprise Agentic AI with innovators from @Accenture@BNY@Comcast@Stanford@QBE & others - 📊 An upcoming series of benchmark datasets and model artifact releases
👀 Want early access to the full agentic AI dataset? Retweet this post and we'll send you the link!
- AI Co-scientist - our OG @GoogleDeepMind Gemini agents for accelerating scientific discovery and helping finding cures for complex diseases (acute myeloid leukemia, liver fibrosis and counting)
- AMIE - our research AI doctor system with Nature papers and real-world clinical deployment
More soon :)
Hazy Research: Strip Mall AI Research Club reposted
Part of my speech at the Columbia Medical Center PhD hooding ceremony:
“The world we’re graduating into now has machines that review literature, write code, analyze data, and even draft manuscripts that pass peer review. Our scientific profession is shifting beneath our feet. Where does this leave the graduating class? I suggest we focus on three things: deep domain knowledge, scientific taste, and character.
First, deep domain knowledge. AI can answer general questions well, but when you go deep, you find that the AI is wrong often, and sometimes in fundamental ways. The world will soon be flooded with AI-generated content. As PhDs, we become world experts in our domain, among the few able to detect subtle AI hallucinations. We can also be a part of the solution, if we so choose. We can help shape AI to better serve our fields.
Second, scientific taste. People talk about the “AI scientist”—a machine that reads papers, spots gaps, and proposes new experiments. This is starting to happen. We shouldn’t compete, but collaborate. As PhDs, our edge lies in intuition: we can guide AI more directly and elegantly toward the discoveries. Such collaboration will also be cheaper and consume less datacenter energy. Remember: we know many things that the internet doesn’t. Unpublished results, failed experiments, and informal conversations over the years. We may better know what to try next and what to avoid. Third, is our character. In a world where every scientist has access to superintelligence, what sets us apart is not what we know, but how we lead. Memorizing the literature and crunching numbers will not be our competitive edge. The scientists who shape the future will be the ones who can guide teams, make good judgments, and act with clarity.”
Hazy Research: Strip Mall AI Research Club reposted
For a rare look into how LLMs are really built, check out @dlwh's retrospective on how we trained the Marin 8B model from scratch (and outperformed Llama 3.1 8B base). It’s an honest account with all the revelations and mistakes we made along our journey. Papers are forced to hide the mess, but the real science happens in the process. marin.readthedocs.io/en/latest/reports/marin-8b-retro/
Hazy Research: Strip Mall AI Research Club reposted
After many months of our team being heads down building, I’m excited to finally share that SIWA, our public testnet for decentralized AI development, is finally live!
This isn’t just a launch. It’s the beginning of something much bigger.
We believe the future is AI-driven. It’s transforming how we live, work, and create—unlocking new paradigms across industries. But like the internet in its early days, the true impact of AI depends on how accessible and open it is to all.
That’s why we started Sahara AI: to accelerate this future and make sure it benefits everyone. AI is not just a tool—it’s a new source of innovation, productivity, and even income. Think of it as a digital oil field, Sahara provides the tools, infrastructure, and pathways for anyone—creators, developers, and enterprises—to tap into it and build the future themselves.
SIWA is the first public step toward that vision. It’s an open testnet that allows AI developers and resource providers to register AI assets, track provenance and attribution, and (eventually) support licensing, revenue sharing, and collaborative agent development. We’ve been building quietly for a while now, with millions of on-chain accounts and hundreds of thousands of real users on our private testnet. Now, we’re opening things up and inviting the broader ecosystem to help us test, iterate, and shape what comes next.
And this is just the beginning. Soon, we’ll be releasing our full AI Developer Platform and AI Marketplace. These platforms are purpose built to be chain-agnostic, to work across web2 and web3. We’re not here to compete with other chains. We’re here to make it easier to develop AI wherever you are, with whatever tools you already use.
If you’re building in AI, Web3, or somewhere in between, this is your invitation to join us! If you’re interesting in beta-testing our AI Dev Platform or AI Marketplace, don’t hesitate to reach out!
SIWA is our public testnet for decentralized AI development and the first real step toward an AI-driven future that’s open, equitable, and accessible to all.
Here’s what that means 🧵👇
Hazy Research: Strip Mall AI Research Club reposted
What would truly open-source AI look like? Not just open weights, open code/data, but *open development*, where the entire research and development process is public *and* anyone can contribute. We built Marin, an open lab, to fulfill this vision:
Hazy Research: Strip Mall AI Research Club reposted
Excited about this work from my group. Optimizing data mixtures is key to efficient training, but it hasn’t been clear (1) how to set up the mixture components and (2) how to avoid crushing overhead. What's the solution?
Online data mixing reduces training costs for foundation models, but faces challenges: ⚠️ Human-defined domains miss semantic nuances ⚠️ Limited eval accessibility ⚠️ Poor scalability
Introducing 🎵R&B: first regroup data, then dynamically reweight domains during training!
Hazy Research: Strip Mall AI Research Club reposted
In our new paper on multimodal AMIE, we're pushing conversational diagnostic AI beyond text to handle images such as skin photos, ECGs, and clinical docs, which provide crucial context in healthcare.
How do we make an AI reason like a clinician during a dynamic, multimodal conversation? One of our key contributions is multimodal state-aware reasoning, built on @GoogleDeepMind Gemini 2.0 Flash.
Instead of just reacting turn-by-turn, AMIE maintains an internal "understanding" of the consultation: ✅ What is known about the patient? ✅ What are the likely diagnoses? ✅ What information (text or visual) is missing?
This internal state allows AMIE to: 👉 Intelligently guide the conversation through phases like history-taking & diagnosis. 👉 Strategically ask for relevant images (like skin photos or screenshots of ECGs/docs) when its internal state shows uncertainty. 👉 Accurately interpret multimodal data and weave the findings back into the ongoing dialogue and diagnostic process.
Essentially, it mimics the adaptive reasoning clinicians use, leading to a more structured and effective consultation.
We evaluated multimodal AMIE against primary care physicians (PCPs) in a demanding, blinded OSCE study using 105 diverse multimodal scenarios.
The results demonstrate clear progress: AMIE achieved similar or superior performance when compared to PCPs across a wide range of metrics, including diagnostic accuracy, empathy, and critically, the handling and reasoning about multimodal data.
While the OSCE results are very promising, it's important to remember this was a test environment with patient actors! Real-world care is more complex. Making sure it's safe, reliable, and actually helpful in the real world needs more work, starting with our upcoming study with Harvard BIDMC.
Building on Articulate Medical Intelligence Explorer — AMIE, our research diagnostic conversational AI agent — today on the blog we share a first of its kind demonstration of a multimodal conversational diagnostic AI agent, multimodal AMIE. Learn more →goo.gle/42D0QcB
Hazy Research: Strip Mall AI Research Club reposted
We studied (and will now talk about) how these ideas let us:
- Take existing Transformer LLMs, and turn them into SoTA subquadratic LLMs
- Get SoTA quality, despite only training 0.2% of past methods' model parameters, with 0.4% of their training tokens (i.e., a 2500x boost in token-efficiency)
We studied (and will now talk about) how these ideas let us:
- Take existing Transformer LLMs, and turn them into SoTA subquadratic LLMs
- Get SoTA quality, despite only training 0.2% of past methods' model parameters, with 0.4% of their training tokens (i.e., a 2500x boost in token-efficiency)
Super excited to share Chipmunk 🐿️- training-free acceleration of diffusion transformers (video, image generation) with dynamic attention & MLP sparsity!
Led by @austinsilveria, @SohamGovande - 3.7x faster video gen, 1.6x faster image gen. Kernels written in TK ⚡️🐱 1/
Training-free acceleration of Diffusion Transformers with dynamic sparsity and cross-step attention/MLP deltas--collaboration with @SohamGovande and @realDanFu!
⚡️ 3.7x faster video and 1.6x faster image generation while preserving quality!
🧵 Open-source code & CUDA kernels!
Hazy Research: Strip Mall AI Research Club reposted
💎 gemma3 qat support — new models from @googleaidevs and @GoogleDeepMind are here, and they ❤️ edge devices! use them in minions today, brought to you by our faves @ollama and mlx (ty @awnihannun)
see how to get started 👇🏽
Hazy Research: Strip Mall AI Research Club reposted
Today, we announced a $200M round led by Addition, @a16z, @lightspeedvp, and Valor, with participation from existing investors @ThriveCapital, Altimeter, Trust Ventures, and @terraincap.
Addition founder Lee Fixel will join our board alongside Antonio Gracias of Valor.
Hazy Research: Strip Mall AI Research Club reposted
.@togethercompute we are thrilled to be the launch partner for @AIatMeta's new SOTA Llama-4 models. These are designed to provide the multi-modal capabilities efficiently (ie low price). Available immediately!
We’re thrilled to announce the launch of Llama 4 models on Together AI.
As a Meta launch partner, we now support the two new groundbreaking multimodal models on the Together API – Llama 4 Maverick and Llama 4 Scout.
Link & details below ⤵️
Hazy Research: Strip Mall AI Research Club@HazyResearch
The Great American AI Race. I wrote something about how we need a holistic AI effort from academia, industry, and the US government to have the best shot at a freer, better educated, and healthier world in AI. I’m a mega bull on the US and open source AI. Maybe we’re cooking something bigger… stay tuned or contact us.
🏎️ samba ftw — your gru just got turbo’d. 250+ t/s cloud speeds now live w/the new @SambaNovaAI client! 🐋 deepseek-v3-0324 — get started with @deepseek_ai’s latest and greatest today 💪🏻
see deepseek-v3-0324 help explain ai biology from @AnthropicAI 👇
Hazy Research: Strip Mall AI Research Club reposted
Comparing DeepSeek V3-0324 APIs: We are now tracking 10 APIs for DeepSeek’s new model, including DeepSeek’s first-party API and offerings from Fireworks, DeepInfra, Hyperbolic, Nebius, CentML, kluster.ai, Novita, Replicate and SambaNova
DeepSeek V3-0324 is open weights and there is now a healthy ecosystem of providers offering APIs! Congrats to all the inference providers for making it rapidly available. Given V3-0324’s status as the leading non-reasoning model, we will not be surprised to see more providers make this model available in the coming days.
Key info from our benchmarking: ➤ We’re seeing the fastest output speeds on SambaNova (267 tokens/s), Fireworks (82 tokens/s) and CentML (76 tokens/s) ➤ We’re seeing the best prices on DeepSeek’s own API ($0.27/$1.1 per million input/output tokens), followed closely by DeepInfra ($0.4/0.89) ➤ Not all providers are supporting V3-0324’s full 128K context window: notably, DeepSeek’s own API only supports a 64K token context window (as does Novita’s API) and SambaNova currently only supports an 8K token context window
DeepSeek V3 and R1 are a particularly hard models to host compared to most other open weights models because they are so large - at 671B total parameters, they cannot fit on a single 8xH100 node. It can be very expensive for providers to offer APIs at small scale.
Hazy Research: Strip Mall AI Research Club reposted
BASED ✌️ turns 1! One year since its launch at NeurIPS 2023 — and it's helped shape the new wave of efficient LMs. ⚡️ Fastest linear attention kernels 🧠 405B models trained on 16 GPUs 💥 Inspired Mamba-v2, RWKVs, MiniMax Checkout our retrospective below!
Hazy Research: Strip Mall AI Research Club reposted
Immediate, fast access to amazing OSS models like DeepSeek-671B, Llama, Flux and more. In the spirit of open access, Together Chat is available will always provide a healthy amount of free access to SOTA OSS AI.
SambaNova has released their new DeepSeek R1 endpoint and is achieving >200 output tokens/s on their first-party RDU chips!
We now benchmark over 15 providers of DeepSeek's R1 model and are seeing substantial differentiation between their offerings: ➤ @SambaNovaAI stands out as the fastest (though offers a 16k context window) ➤ @DeepInfra is offering an endpoint with FP-4 precision at almost half the price of their native FP-8 precision endpoint ➤ @deepseek_ai remains the cheapest provider but offers significantly slower output speeds than other providers. We have also measured substantial variance in time to first token on DeepSeek's first-party API. This is potentially due to queuing of requests to manage infra load.
See below for an analysis of the Output Speed vs. Price of DeepSeek R1 providers we benchmark 👇
Hazy Research: Strip Mall AI Research Club reposted
Llama has now been downloaded over 1 Billion times!
A note to: The researchers at Meta training these models — and those building on the research in other labs. The developers and enthusiasts on r/LocalLlama, @huggingface and more; experimenting with new models and creating derivatives. The small startups and big enterprises alike who are creating a new wave of AI-powered products, built with Llama. The global AI community.
Your actions speak louder than words, thank you for making it abundantly clear — a billion times over — that open source AI is how we'll create the next wave of world changing technologies, together. 🦙❤️
Hazy Research: Strip Mall AI Research Club reposted
🌿power consumption monitoring! — we shipped a v0 minions workload power monitor so you can track energy savings vs. cloud-only setups. minions ❤️ the 🌍
Wrapping up our trio of decode kernels, we’re excited to announce ThunderMHA!
Our fused decode kernels pack now supports Multi-Head Attention (MHA), powering even faster inference for day-1 architectures like Transformers, GPT, and BERT.
10%+ faster than FA3 on H100s, we’re excited to keep on pushing perf 🚀.
(1/7) Inspired by DeepSeek's FlashMLA, we're releasing ThunderMLA—a fused megakernel optimized for variable-prompt decoding! ⚡️🐱ThunderMLA is up to 35% faster than FlashMLA and just 400 LoC.
My latest on the importance of American leadership in open-source AI. The chip war and the open source race are merging. Given the stakes, second place is not an option.
How can we use small LLMs to shift more AI workloads onto our laptops and phones?
In our paper and open-source code, we pair on-device LLMs (@ollama) with frontier LLMs in the cloud (@openai, @together), to solve token-intensive workloads on your 💻 at 17.5% of the cloud cost while maintaining 97.9% of the accuracy.
See Gru and the Minions in action below, 🔉on please (h/t @cartesia)!
We joined @MongoDB! @VoyageAI’s best-in-class embedding models and rerankers will be part of MongoDB’s best-in-class database, powering mission-critical AI applications with high-quality semantic retrieval capability.
A huge thank you to everyone with us on this journey, and to @dittycheria and @sahirazam for their vision in redefining the database industry for the AI era.
Incredibly grateful and honored to receive the 2025 Overton Prize♥️
Surreal to follow the steps of my science heroes🙏 Truly, credit goes to my amazing students, collaborators and mentors who make research so fun + inspiring!
Also underscores the value of supporting open academic environment that make this possible. Thank you @iscb!
Hazy Research: Strip Mall AI Research Club reposted
I received many warmhearted messages from friends and colleagues these days--thank you, everyone. Receiving this award from @SloanFoundation still feels unreal.
I almost didn't apply in the midst of an extremely difficult 2024 dealing with personal challenges. But I want to thank Tom Reps and our department for encouraging me to apply.
I would like to particularly thank my respected colleagues at @UWMadison: Jerry Zhu, Steve Wright @madsjw, Rob Nowak @rdnowak, and Amos Ron for guiding and supporting my applications. I’m also immensely grateful to my PhD advisor, John Hopcroft, and postdoc advisor, Chris Ré (@HazyResearch), whose mentorship laid the foundation for my career.
To my students and collaborators: this is a shared milestone. Your passion, creativity, and dedication continue to inspire me, and I look forward to all we will achieve together.
🎉Congrats to the 126 early-career scientists who have been awarded a Sloan Research Fellowship this year! These exceptional scholars are drawn from 51 institutions across the US and Canada, and represent the next generation of groundbreaking researchers. sloan.org/fellowships/2025-Fellows
Hazy Research: Strip Mall AI Research Club reposted
[1/7] Introducing Evo 2, a new foundation model for biology. 🚀 Evo 2 is the largest-scale, fully open-source AI model ever released: 40 billion parameters, over 9 trillion tokens, and a 1 million context length. All the details are public: weights, data, training infrastructure, and inference infrastructure.
⚡Evo 2 is built on a new model architecture: convolutional multi-hybrids (StripedHyena 2). StripedHyena 2 excels at modeling byte-tokenized data, providing faster training and lower perplexity compared to both Transformers and previous-generation hybrids based on state-space models.
I am grateful for the team behind Evo 2—working with you was one of the proudest moments of my career (the core pretraining team was fewer than five people; you can just do things).
📚 Today, we release two papers (yes, plural), as well as weights, data, training, and inference codebases. Enjoy!
Hazy Research: Strip Mall AI Research Club reposted
🏎️⚡️Ka-chow⚡️ The fastest DeepSeek-R1 671B on SambaNova Cloud — running at 198 t/s!
✅3X faster & 5X more efficient than the latest GPUs ✅Running on 1 rack efficiently (16 RDUs) ✅Hosted in secure US data centers ✅100X the global capacity by the end of 2025
Very happy to show that we can do RL finetuning on 405B models with open-source code, beat Llama 405B instruct with their base model, and beat DeepSeek V3 too. Enjoy building off this teams hard work.
Here's Tulu 3 405B. A holiday present from @hamishivi, @vwxyzjn and team.
Here is Tülu 3 405B 🐫 our open-source post-training model that surpasses the performance of DeepSeek-V3! The last member of the Tülu 3 family demonstrates that our recipe, which includes Reinforcement Learning from Verifiable Rewards (RVLR) scales to 405B - with performance on par with GPT-4o, and surpassing prior open-weight post-trained models of the same size including Llama 3.1
Hazy Research: Strip Mall AI Research Club reposted
Excited to release our newest, largest, and best Tulu yet. Our RLVR recipe works at scale, outperforming Deepseek V3. So proud of the team! And @hamishivi@vwxyzjn for scaling up the Tulu recipe.
Here is Tülu 3 405B 🐫 our open-source post-training model that surpasses the performance of DeepSeek-V3! The last member of the Tülu 3 family demonstrates that our recipe, which includes Reinforcement Learning from Verifiable Rewards (RVLR) scales to 405B - with performance on par with GPT-4o, and surpassing prior open-weight post-trained models of the same size including Llama 3.1
Hazy Research: Strip Mall AI Research Club reposted
Ten years ago today, Sergey and I finished training some of the first end-to-end neutral nets for robot control 🤖
We submitted the paper to RSS on January 23, 2015.
It was rejected for being "incremental" and "unlikely to have much impact"
Our resubmission to NeurIPS was also rejected
It now has >4,000 citations (and more importantly, end-to-end training is widely accepted!)
It's also cool to think about what's changed and what's the same -- - The network was 92k parameters and trained on ~15 minutes of data - The code was a combination of matlab, caffe, ROS, a custom CUDA kernel for speed, and a low-level 20 Hz controller in C++, all talking to each other. ROS+matlab was as bad as it sounds. - We pre-trained the encoder and did inference off-board on a workstation with a larger GPU. - We were paranoid about varying lighting messing up the network, so we did all the experiments after sunset (so long nights running experiments on the robot past 3 am)
Now, we have manipulation policies that are far more dextrous, far more generalizable, and maybe on the cusp of breaking into the real world. :)
Our approach to evaluating health AI models continues to evolve!
(Phase 1) Medical Benchmarks ➡️ (Phase 2) Patient Actor Consultations ➡️ (Phase 3 – coming soon!) Real-World Deployment
> (Phase 1) Medical Benchmarks: We first need to make sure our models have extensive medical knowledge and clinical reasoning ability. To do this, with Med-PaLM and Med-Gemini, we focused on evaluating our health AI models on a broad range of medical exams and established benchmarks [1]. > (Phase 2) Patient Actor Consultations: We then need to take a step closer to real-world deployment. With AMIE, we ran studies with patient actors to understand how our health AI models perform in more realistic, conversational settings [2]. > (Phase 3) Real-World Deployment: Coming next, in partnership with Harvard @BIDMChealth, we're launching a prospective study to evaluate AMIE in a real-world clinical setting, exploring its impact on pre-visit information gathering and the perceptions of both clinicians and patients, under strict medical supervision [3].
Excited to report back what we learn! Check out our @GoogleAI blogs below for more info.
Leaps in AI could have transformative benefits for health & medicine. Delighted to share updates on AMIE - our bold & responsible research towards Conversational Diagnostic AI. New results in specialist topics like oncology/cardiology; and a new partnership with @BIDMChealth for prospective real-world validation in 2025. More in our @GoogleAI blog 👇
@simran_s_arora from @HazyResearch on GPU kernel generation, project 🍿and the challenge of AI mapping algorithms to hardware architecture. At @NeurIPSConf Hacker AI Cup
Hazy Research: Strip Mall AI Research Club reposted
We have acquired @CodeSandbox, a pioneer in code execution environments, and are joining forces to launch Together Code Interpreter—giving LLMs the ability to seamlessly execute the code they write.
Here’s why this is a big deal for AI developers 👇
Hazy Research: Strip Mall AI Research Club reposted
We’re thrilled to lead $27M in new funding for @cartesia as they build the next generation of real-time AI. Their pioneering SSMs offer multimodal intelligence available on any device, powering new solutions for customer service, healthcare, transportation, robotics, and more.
Excited to share that Quora is partnering with Cartesia to add audio capabilities to Poe! ✨
Users can already interact with advanced LLMs on the @poe_platform. Now they can also get audio responses in diverse voices and languages, transforming text into audio for a richer experience.
Want to hear today's news in our vintage 1920's Radioman voice, or create epic stories narrated by our immersive Wizardman voice? You can do it all on Poe.
"Poe brings together the world's best AI, all in one place. With Cartesia's Sonic model, users can interact with a wide range of high-quality, human-like voices in multiple languages, enhancing their experience on our platform," says Spencer Chan, Quora's Head of Poe Product.
Hazy Research: Strip Mall AI Research Club reposted
We've been trying out Devin & were very impressed with what it was able to do.
One of our engineers gave it a task to build a K8S operator with a technical spec and some scripts, and it eventually delivered a very well written operator, testing environment, and build script
Today in @Nature, we’re presenting GenCast: our new AI weather model which gives us the probabilities of different weather conditions up to 15 days ahead with state-of-the-art accuracy. ☁️⚡
Kernels are the kernel of deep learning. 🙃...but writing kernels sucks. Can LLMs help? 🤔
Introducing 🌽 KernelBench (Preview), a new coding benchmark designed to evaluate the ability of LLMs to generate ⚡️efficient💨 GPU kernels for optimizing neural network performance.
Hazy Research: Strip Mall AI Research Club reposted
At the CUDA Mode IRL hackathon, @anneouyang and I tried this dream of ours as performance engineers: Can we get LLMs to go CUDA Mode 🧑💻 and write CUDA kernels?
Code → ⚙️ LLM → Optimized ⚡ Code, that goes brrrrrr 💨 on GPUs?
As much as I enjoyed learning and writing custom kernels for LLMs @cohere and for fun (making PRs to llm.c), it is incredibly hard to master CUDA knowledge 🥷 and super time-consuming 🐢 to write kernels for every new operators/hardware/input dimensions. 😵💫
Just as @karpathy talked about in his keynote, what if we can have a 🔮 “llm.c”, a set of custom kernels and optimized code, for any model architecture one like to optimize for?
Language models might be a piece of the puzzle 🧩 here. We designed KernelBench to see how far current LLMs can optimize PyTorch code with correct ✅ and performant 🏁 custom CUDA kernels.
We are surprised by some of the optimizations LLMs were able to generate, but also how much more room for improvement there could be!
A small step towards a future 🔭 where we can quickly create custom, performant software for any application.
Kernels are the kernel of deep learning. 🙃...but writing kernels sucks. Can LLMs help? 🤔
Introducing 🌽 KernelBench (Preview), a new coding benchmark designed to evaluate the ability of LLMs to generate ⚡️efficient💨 GPU kernels for optimizing neural network performance.
Hazy Research: Strip Mall AI Research Club reposted
We're thrilled to congratulate our cofounders @bclyang and @jundesai for their recognition on the @ForbesUnder30 AI list! Their leadership and vision continue to drive our mission to build the best real time AI models.
Hazy Research: Strip Mall AI Research Club reposted
Thankful for TK kernels that now run on Apple hardware - with @CKT_Conner, bringing faster attention and GEMMs to your M2 MacBooks! happy thanksgiving 🍁
Hazy Research: Strip Mall AI Research Club reposted
this award reminds me of how GAN started with this one email ian sent to the @Mila_Quebec lab mailing list in May 2014. super insightful and amazing execution!
Hazy Research: Strip Mall AI Research Club reposted
We are excited to announce the launch of our company - Cognita! We are working towards building the future of radiology through multi-modal AI systems with a great group of founders @loublanks , @zhjohnchan, and I, and advisors Ajit Singh, Chris Re, and @curtlanglotz
1/2
Hazy Research: Strip Mall AI Research Club reposted
happy thanksgiving!! we're excited to share new thunderkittens blog posts this week for your enjoyment over the holiday break. today, we're starting off with fp8 support!!! 1500 tflops in <95 lines of code 🔥
Hazy Research: Strip Mall AI Research Club reposted
We built a B2B SaaS sales company and here’s what it taught us about B2B SaaS sales 🧵👇 (but actually)
Today we’re launching Rox, the first publicly available AI agent swarm for the top sales teams, and in the private beta it already helped reps grow their books 30%.
2025 is going to be a huge year for growth. Enterprises are doubling next year’s revenue goals, but no one is doubling team sizes. Every rep will have to bring in more, and AI can help them do that. But there’s a wrong way and a right way.
Much of today’s AI aims to replace low-value work. But sales follows a power law: 90% of revenue comes from the top 15% of enterprise sales reps. The greatest gains will come from supercharging the highest value work — raising the ceiling, not the floor.
Rox equips the very best with a swarm of AI agents, acting as an army of analysts to help them plan, prioritize, research, engage, and keep up with their customers.
Over 35 of the best-performing enterprise sales teams have adopted Rox virally. For example, Ramp has rolled it out to their AE and AM teams, and we are now integrating their internal data systems with Rox. The Enterprise AE team alone gains 225+ hours per week to boost pipeline execution activities.
Rox is now in public beta. No barriers. No need to request a demo. Try it now for free → rox.com
Excited to share how Cartesia is transforming healthcare communication with Hello Patient!
Even as medical practices embrace digital transformation, patients still overwhelmingly prefer picking up the phone. After years of leading patient-facing products and rebuilding call center infrastructure at Carbon Health, @anothercohen founded Hello Patient to revolutionize how medical practices handle patient communications.
During @HelloPatient's stealth phase, Alex saw the potential in Cartesia's technology.
As he shares: "At Carbon Health, we were handling 30,000 calls per day, with support costs consuming a significant part of revenue. When we discovered Cartesia, I immediately saw the potential to build what we always wished we had - natural-sounding AI voice technology that could actually handle complex patient conversations."
That Chip Has Sailed: A Critique of Unfounded Skepticism Around AI for Chip Design arxiv.org/abs/2411.10053
We pioneered agentic deep reinforcement learning for chip layout optimization in 2020. Our method, AlphaChip, has now been used in production in advanced chips across Alphabet and externally, including three generations of TPUs and CPUs. These chips are now running in datacenters and devices all over the world.
Our paper laid the groundwork for a significant subsequent body of research in AI for chip design. External academics and chipmakers have announced work that builds on AlphaChip and extends it to other areas of chip design like macro selection, logic synthesis and top-level floorplanning. Major EDA and chip design companies have announced their use of reinforcement learning to address a wide variety of challenging optimization problems.
Even so, an individual continues to raise doubts about our work in the EDA community. While AlphaChip continues to inspire new work in AI for chip design, we wrote this response paper to ensure that no one is discouraged from working in this increasingly crucial and impactful area.
This post explains why we started the work that led to Evo (HyenaDNA), recently on the cover of Science–thanks to a host of wonderful collaborators at @arcinstitute . It has some odd musing about foundation models and science... and what I got and continue to get wrong about foundation models, @StanfordCRFM.
H/t @mzhangio for the memes. If you suffer to the end, this not very funny joke will make more sense... maybe. It's long.
Hazy Research: Strip Mall AI Research Club reposted
“How has multimodality shaped your core approach at Cartesia?”
Our work on State Space Models is about creating universal building blocks for multimodality. While we're demonstrating capabilities across text, audio, and video (starting with Sonic), we see different modalities as unique implementation challenges rather than architectural drivers.
Cartesia's core technology is modality-agnostic – fundamental components that can be efficiently adapted to any domain, with multimodal applications being just one expression of these universal building blocks.
We’re excited to pursue this path and shape the future of voice AI to start.
Hazy Research: Strip Mall AI Research Club reposted
SambaNova extends its Llama 3.1 405B inference speed lead - achieving 163 output tokens/s, >2X other providers
@SambaNovaAI has rolled out a speculative decoding on their 405B endpoint, now delivering speeds ranging up to 200 tokens/s (depending on prompt complexity).
As we’ve covered previously, speculative decoding is an inference optimization technique that uses a smaller draft model to generate speculative tokens for an LLM to verify. Speculative decoding does not impact quality when implemented with the full model verifying all output tokens. One implication of speculative decoding is that ‘simpler’ prompts can run faster than ‘harder’ prompts because a higher proportion of the draft model’s output tokens are accepted as correct by the target model.
We have independently benchmarked SambaNova’s updated endpoint and can confirm that we have observed no quality degradation in the latest version of the API.
See below for an over time view and for a link to our analysis of providers of Llama 3.1 405B 👇
Hazy Research: Strip Mall AI Research Club@HazyResearch
Thank you to @arcinstitute and the great @BrianHie for teaching us so much and taking a chance on a group that insists on naming its models after zoo animals (hyenaDNA, (hungry or not!) hippos, mamba) ... and its odd choices in AI architectures too.
This project wouldn't have happened unless the effort was led by brilliant students @MichaelPoli6 and @exnx, who took a risk on training a DNA model with an advisor who finds the human body to be disgusting (beings of pure energy would be way more fun!). They wisely found better collaborators, built something amazing, and let me peak in... The whole lab is incredibly proud of them.
These new state space models are fun, and so many in our lab have contributed to this line of work. We're so close to the beginning of the AIs we can build. Excited for what's next!
A new Science study presents “Evo”—a machine learning model capable of decoding and designing DNA, RNA, and protein sequences, from molecular to genome scale, with unparalleled accuracy.
Evo’s ability to predict, generate, and engineer entire genomic sequences could change the way synthetic biology is done. Learn more in this week's issue: www.science.org/toc/science/386/6723
Hazy Research: Strip Mall AI Research Club@HazyResearch
Thank you also to @StanfordHAI for supporting this and on-going work.
Hazy Research: Strip Mall AI Research Club reposted
Evo has been published in @Science! A true privilege to work with such an amazing team!
So many exciting new experimental results in this emerging field of Generative Genomics, including AI generated and *validated* CRISPR-Cas systems and transposons
A new Science study presents “Evo”—a machine learning model capable of decoding and designing DNA, RNA, and protein sequences, from molecular to genome scale, with unparalleled accuracy.
Evo’s ability to predict, generate, and engineer entire genomic sequences could change the way synthetic biology is done. Learn more in this week's issue: www.science.org/toc/science/386/6723
Hazy Research: Strip Mall AI Research Club@HazyResearch
Thank you to @arcinstitute and the great @BrianHie for teaching us so much and taking a chance on a group that insists on naming its models after zoo animals (hyenaDNA, (hungry or not!) hippos, mamba) ... and its odd choices in AI architectures too. This project wouldn't have happened unless the effort was led by brilliant students @MichaelPoli6 and @exnx, who took a risk on training a DNA model with an advisor who finds the human body to be disgusting (beings of pure energy would be way more fun!). They wisely found better collaborators, built something amazing, and let me peak in... The whole lab is incredibly proud of them.
These new state space models are fun, and so many in our lab have contributed to this line of work. We're so close to the beginning of the AIs we can build. Excited for what's next!
Hazy Research: Strip Mall AI Research Club reposted
A new Science study presents “Evo”—a machine learning model capable of decoding and designing DNA, RNA, and protein sequences, from molecular to genome scale, with unparalleled accuracy.
Evo’s ability to predict, generate, and engineer entire genomic sequences could change the way synthetic biology is done. Learn more in this week's issue: www.science.org/toc/science/386/6723
Hazy Research: Strip Mall AI Research Club reposted
There are many algorithms for constructing pre-training data mixtures—which one should we use? Turns out: many of them fall under one framework, have similar issues, and can be improved with a straightforward modification.
Introducing Aioli! 🧄 1/9
Hazy Research: Strip Mall AI Research Club reposted
(1/7) In celebration of National Cat Day, we’re excited to release our first major batch of updates to ThunderKittens! ThunderKittens is now easier, better, faster, and cuter than ever before! In addition to massive speed boosts, we’re releasing a broad swath of kernels, new features, and tons of usability improvements.
(1/7) In celebration of National Cat Day, we’re excited to release our first major batch of updates to ThunderKittens! ThunderKittens is now easier, better, faster, and cuter than ever before! In addition to massive speed boosts, we’re releasing a broad swath of kernels, new features, and tons of usability improvements.
the transformer is good for big tech because it means the only way to win in AI is to have the biggest data center
the bitter lesson matters, but yelling about it with suboptimal asymptotics is bad…
the incumbents aren’t incentivized to change this, which is where we come in
if models require huge resources to train and inference, it creates lockout for the gpu poors
forget open or closed source, this is just bad for competition, and disrupting the efficiency of architectures is an imperative to remove the monopolies
architectural anarchy is the future
new training and inference algorithms to invent, new domains to conquer, and new products to build
Hazy Research: Strip Mall AI Research Club reposted
🚀 Exciting news! We're proud to announce OccamLLM, a novel method for exact language-model arithmetic in a single autoregressive step—no code generation or LLM finetuning needed! 🧮 OccamLLM has been accepted to NeurIPS 2024!
Hazy Research: Strip Mall AI Research Club reposted
While we started with simple linear attentions, we hope that our findings + code can help you linearize your Llamas, Mistrals, Gemmas or whatevers into the architectures of your hearts’ desires.
Hazy Research: Strip Mall AI Research Club reposted
Ever wanted to scale subquadratic models up to 7B+ LLMs? But didn't want to pretrain billions of parameters on trillions of tokens?
Then just for you, we're happy to share LoLCATs 😺
We show how to convert existing Transformers like Llama 3 8B & Mistral 7B into state-of-the-art subquadratic LLMs
But now for the same cost as a LoRA finetune! (Train just 0.2% of an LLM's parameters, on 0.04-0.2% of prior conversion methods’ training tokens; 0.003% that of pretraining from scratch)
And then kind of just because we could, used LoLCATs to make the first linearized 70B and 405B LLMs (🙏 @AIatMeta for Llama 3.1)
What is the best way to spend your inference compute budget to create LLM systems greater than the sum of their parts?
In our latest paper, we present Archon, an architecture search framework for inference-time techniques! Archon is enabled by inference-time architecture search (ITAS), allowing us to find the best combination of LLMs and inference-time techniques!
With Archon, we outperform GPT-4o and Claude-3.5-Sonnet single-call win-rate/accuracy by an average 14.1 percentage points across instruction-following, reasoning, and coding tasks! And when only using open-source 70B+ models, we can outperform GPT-4o and Claude-3.5-Sonnet single-call by 10.3 percentage points!
Users come to us all the time with questions around how to evaluate the best voice generation APIs. To help, we put together a systematic comparison on the important features to look at when comparing Cartesia Sonic to ElevenLabs (link below)
Another great resource is the @ArtificialAnlys Text-to-Speech Arena, which conducts a blind human preference test on voice quality across providers.
Hazy Research: Strip Mall AI Research Club reposted
The ecosystem around Llama is continuing to push the limits. SambaNova Cloud is setting a new bar for inference on 405B and it’s available for developers to start building today.
Today, we’re unveiling a significant milestone in our journey toward ubiquitous artificial intelligence: AI On-Device.
Our team pioneered a radically more efficient architecture for AI with state space models (SSMs). Now, we’ve optimized and deployed them at the edge. We believe the future of AI runs on your device, where it can process continuously and is reliable, private, and secure.
📄Jamba-1.5 whitepaper is out! The whitepaper details the architecture, training schemes, novelties and in-depth evaluations of our new long context hybrid SSM-Transformer models - Jamba-1.5-Large and Jamba-1.5-Mini.
Excited to share that I will be joining UCSD CSE as an assistant professor in January 2026!
I'll be recruiting PhD students from the 2024 application pool - if you're interested in anything ML Sys/efficiency/etc please reach out & put my name on your application!
Until then I'll be finishing up some requirements at Stanford (long story...) and hanging out at @togethercompute. Stay tuned for more!
Hazy Research: Strip Mall AI Research Club reposted
We are pleased to announce six @Stanford research teams receiving this year's Hoffman-Yee Grants, a key initiative of @StanfordHAI since our founding. Read about the scholars who are ready to shape the future of human-centered AI: stanford.io/3MdGMEz
Hazy Research: Strip Mall AI Research Club reposted
- 256K #contextwindow - Up to 2.5X faster on #longcontext in its size class - Native support for structured JSON output, function calling, digesting doc objects & generating citations
Do you like LLMs? Do you also like for loops? Then you’ll love our new paper!
We scale inference compute through repeated sampling: we let models make hundreds or thousands of attempts when solving a problem, rather than just one. By simply sampling more, we can boost LLM performance across a range of math and coding tasks, allowing weaker models (ex. Llama-3-8B) to outperform single attempts from more capable models (ex. GPT-4o).
Notably, with DeepSeek-Coder-V2-Instruct and 250 attempts, we solve 56% of issues from SWE-bench Lite, outperforming the single-attempt SOTA of 43%.
🚀 World record performance: SambaNova is running Llama 3.1 405B at 114 t/s with full precision accuracy, in only one rack. Verified by @ArtificialAnlys! 🦙
This speed unlocks so many use cases for enterprises and developers that we cannot wait to see them built on our platform.
SambaNova is serving Llama 3.1 405B at 114 output tokens/s with their custom chips! This is the fastest we have benchmarked and 4X faster than the median of providers on Artificial Analysis.
Larger models with higher quality come at the cost of speed. New AI-focused custom silicon technologies like @SambaNovaAI's custom RDU chips, and others like Groq, are running bigger models faster - allowing developers to use bigger models at similar speeds to what they were getting on smaller models on GPUs. SambaNova has also declared the model is being stored and served at FP16, indicating quality has not been compromised in achieving these speeds.
While SambaNova is not listed on our public leaderboards due to not having an openly accessible endpoint with per token pricing, we understand they are expanding access and per token pricing is coming soon.
See below for a link to their chat interface where you can try it out for yourself, and also a link to where API access can be requested 👇
Hazy Research: Strip Mall AI Research Club reposted
🚀 Excited to partner with Meta to bring Llama 3.1 models to Together Inference and Fine-tuning! Up to 80 tokens per second for Llama 3.1 405B and up to 400 tokens per second for Llama 3.1 8B, which is 1.9x to 4.5x faster than vLLM while maintaining full accuracy.
Hazy Research: Strip Mall AI Research Club reposted
#icml2024 🥳 Will be at MoE tutorial panel today, present 6 papers about LLM efficient training and inference Tue-Thurs, and give invited talks at Long-context modeling and Mem efficient training workshops and co-host two Fri-Sat. Excited to meet people @icmlconf! DM/Email or come and find me in the following locations 🙃🙃🙃
Hazy Research: Strip Mall AI Research Club reposted
Today we are announcing a new inference stack, which provides decoding throughput 4x faster than open-source vLLM.
We are also introducing new Together Turbo and Together Lite endpoints that enable performance, quality, and price flexibility so you do not have to compromise.
Hazy Research: Strip Mall AI Research Club reposted
After months of hard work, our team is thrilled to bring you our #1 most requested feature — Guardrails Server. As more engineers put @guardrails_ai around their LLM in production, we’re making Guardrails deployment easier than ever.
Why this matters: ✅ Easy cloud deployment: Seamlessly dockerize the Guards you’re running locally ✅ OpenAI SDK compatibility: Protect your OpenAI or LLM Proxy service with a single line change. ✅ Cross language support: Use the OpenAI-compatible endpoint in any language which supports the OpenAI SDK.
(1/n)
Hazy Research: Strip Mall AI Research Club reposted
⚡️ Excited to share that I am starting an AI+Education company called Eureka Labs. The announcement:
--- We are Eureka Labs and we are building a new kind of school that is AI native.
How can we approach an ideal experience for learning something new? For example, in the case of physics one could imagine working through very high quality course materials together with Feynman, who is there to guide you every step of the way. Unfortunately, subject matter experts who are deeply passionate, great at teaching, infinitely patient and fluent in all of the world's languages are also very scarce and cannot personally tutor all 8 billion of us on demand.
However, with recent progress in generative AI, this learning experience feels tractable. The teacher still designs the course materials, but they are supported, leveraged and scaled with an AI Teaching Assistant who is optimized to help guide the students through them. This Teacher + AI symbiosis could run an entire curriculum of courses on a common platform. If we are successful, it will be easy for anyone to learn anything, expanding education in both reach (a large number of people learning something) and extent (any one person learning a large amount of subjects, beyond what may be possible today unassisted).
Our first product will be the world's obviously best AI course, LLM101n. This is an undergraduate-level class that guides the student through training their own AI, very similar to a smaller version of the AI Teaching Assistant itself. The course materials will be available online, but we also plan to run both digital and physical cohorts of people going through it together.
Today, we are heads down building LLM101n, but we look forward to a future where AI is a key technology for increasing human potential. What would you like to learn? ---
@EurekaLabsAI is the culmination of my passion in both AI and education over ~2 decades. My interest in education took me from YouTube tutorials on Rubik's cubes to starting CS231n at Stanford, to my more recent Zero-to-Hero AI series. While my work in AI took me from academic research at Stanford to real-world products at Tesla and AGI research at OpenAI. All of my work combining the two so far has only been part-time, as side quests to my "real job", so I am quite excited to dive in and build something great, professionally and full-time.
It's still early days but I wanted to announce the company so that I can build publicly instead of keeping a secret that isn't. Outbound links with a bit more info in the reply!
Hazy Research: Strip Mall AI Research Club reposted
FlashAttention is widely used to accelerate Transformers, already making attention 4-8x faster, but has yet to take advantage of modern GPUs. We’re releasing FlashAttention-3: 1.5-2x faster on FP16, up to 740 TFLOPS on H100 (75% util), and FP8 gets close to 1.2 PFLOPS! 1/
Hazy Research: Strip Mall AI Research Club reposted
Excited to share Just read twice: going beyond causal language modeling to close quality gaps between efficient recurrent models and attention-based models!!
There’s so much recent progress on recurrent architectures, which are dramatically more memory efficient and asymptotically faster than attention 💨 But there’s no free lunch 🥪 these models can’t fit all the information from long contexts into the limited memory, degrading in-context learning quality. Is all lost?
Hazy Research: Strip Mall AI Research Club reposted
A 8B-3.5T hybrid SSM model gets better accuracy than an 8B-3.5T transformer trained on the same dataset: * 7% attention, the rest is Mamba2 * MMLU jumps from 50 to 53.6% * Training efficiency is the same * Inference cost is much less arxiv.org/pdf/2406.07887
Hazy Research: Strip Mall AI Research Club reposted
Mixture of Agents—a framework that leverages the collective strengths of multiple LLMs. Each layer contains multiple agents that refine responses using outputs from the preceding layer. Together MoA achieves a score of 65.1% on AlpacaEval 2.0. together.ai/blog/together-moa
Hazy Research: Strip Mall AI Research Club reposted
Yesterday, @Cartesia released Sonic, a blazing fast lifelike generative voice model and API.
We are thrilled to partner with them to achieve the fastest text-to-voice generations in the market with 135 ms model latency 🚀 to provide real-time inference to their users with high accuracy and low cost at production scale.
Today, we’re excited to release the first step in our mission to build real time multimodal intelligence for every device: Sonic, a blazing fast (🚀 135ms model latency), lifelike generative voice model and API.
Today, we’re excited to release the first step in our mission to build real time multimodal intelligence for every device: Sonic, a blazing fast (🚀 135ms model latency), lifelike generative voice model and API.
Artificial Analysis has independently benchmarked @SambaNovaAI's custom AI chips at 1,084 tokens/s on Llama 3 Instruct (8B)! 🏁
This is the fastest output speed we have benchmarked to date and >8 times faster than the median output speed across API providers of @Meta's Llama 3 Instruct (8B) we benchmark.
SambaNova currently does not yet publicly offer a serverless API but you can try out their system via their chat interface (see below tweet). SambaNova is not yet listed on the Artificial Analysis leaderboard but we understand API services using SambaNova chips will be available in the near future and we look forward to initiating full coverage.
SambaNova’s custom SN40L RDU chips are their fourth generation design and are built on TSMC’s 5nm process. They are reported as having the potential to scale to serve much larger models than Llama 3 Instruct (8B) - Llama 3 400B+ 👀.
Artificial Analysis has also verified that Llama 3 Instruct (8B) on Samba-1 Turbo achieves quality scores in-line with full FP16 precision by testing an MMLU-based benchmark.
Hazy Research: Strip Mall AI Research Club reposted
We've added 6,096 H100s to our fleet in Together Cloud. Leading AI companies like @pika_labs , @cognition_labs and @liquidai train on Together Cloud. We have optimized the infrastructure and software for large scale training and inference. Visit www.together.ai/forms/gpu-cluster-requests to reserve while supplies last!
Hazy Research: Strip Mall AI Research Club reposted
#1arxiv.org/pdf/2403.09603 Popular model training services lack user trust due to possible 😈 intent, but 3rd-party auditing suffers w/ GPU nondeterminism. @simran_s_arora, Dan Boneh, and I control this to build a robust verification scheme 🙂
Hazy Research: Strip Mall AI Research Club reposted
ThunderKittens is a Flash Attention implementation that’s just 100 lines of code and 30% faster, and we’re happy to provide.
It includes a simple DSL embedded within CUDA that makes it easy to express key technical ideas for building AI kernels. TK lets us write clean, easy-to-understand code that maximizes GPU utilization -- on all kinds of kernels!
(1/7) Happy mother’s day! We think what the mothers of America really want is a Flash Attention implementation that’s just 100 lines of code and 30% faster, and we’re happy to provide.
(1/7) Happy mother’s day! We think what the mothers of America really want is a Flash Attention implementation that’s just 100 lines of code and 30% faster, and we’re happy to provide.
Prediction: we'll soon abstract away from the distinction of *how* LLMs are adapted.
Developers will instead focus exclusively on *what* labeled data they adapt their LLMs with.
Whether this data is injected via a prompt, fine-tuning, alignment, etc will become a low-level detail decided automatically according to cost/latency tradeoffs.
We already see a convergence of alignment and fine-tuning with the progression from e.g. RLHF --> DPO/KTO/et al. To the user, the only difference is now really just the *type* of labeled data you add in (e.g. prompt/response vs. preference).
We also see the convergence of prompting and fine-tuning as context windows increase in size (so more labeled data can be injected), new ICL approaches that close the gap with FT are proposes (e.g. arxiv.org/abs/2306.07536), new abstraction layers are built (like the awesome DSPy by @lateinteraction ), and new approaches for unifying prompting and labeling (e.g. arxiv.org/abs/2205.02318 from @SnorkelAI ) are proposed.
In the near future: developers will just focus on encoding the task/behavior they need via developing the right datasets- the rest will be automated & abstracted away!
Hazy Research: Strip Mall AI Research Club reposted
Introducing Med-Gemini, a family of models that extends the best of Gemini into medicine! ✨⚕️
Highlights of what you can do with Med-Gemini: > Answer medical questions with up-to-date knowledge using agentic web search 🔎❤️🩹 > Converse about your medical images, videos, and long multi-visit health records 📷📹📃 > Do a literature search by uploading tens of biomedical papers and asking questions 📚 > And so much more!
🏗️ Development of Med-Gemini included: > Advancing clinical reasoning with self-training and search > Improving multimodal understanding with fine-tuning > Leveraging long-context capabilities with chain-of-reasoning
Delighted to share ✨Med-Gemini✨ - our new family of multimodal models for medicine unlocking new possibilities for health - arxiv.org/pdf/2404.18416
More accurate multimodal conversations about medical images🩻, surgical videos📽️, genomics🧬, ultra-long health records📚, ECGs🫀 & more with state-of-art performance across multiple benchmarks
More accurate, up-to-date answers to medical questions with advanced reasoning and intelligent use of web-search
Long-context abilities. Summaries or referral letters from long health records, analyses of dozens of long research PDFs & more (1/6)
Hazy Research: Strip Mall AI Research Club reposted
SoTA LLMs typically exhibit 99%+ non-zero activations, but it turns out that they are still intrinsically quite sparse!
We introduce CATS, a simple post-training technique that achieves 50% activation sparsity for MLP layers with almost no drop in downstream evals, while requiring little to no finetuning! With our custom GPU kernels, we leverage this sparsity to achieve 15% improvement in end-to-end generation latency for both Mistral-7B and Llama-7B models.
There is a really nice community of researchers developing transformer alternatives. Want to highlight these impressive folks. Simran Arora (@simran_s_arora), Chunting Zhou (@violet_zct), Dan Fu (@realDanFu), and Songlin Yang (@SonglinYang4)
Hazy Research: Strip Mall AI Research Club reposted
🚀🌟🚀Excited to announce Samba-CoE v0.2, which outperforms DBRX by @DbrxMosaicAI and @databricks, Mixtral-8x7B from @MistralAI, and Grok-1 by @grok at a breakneck speed of 330 tokens/s. These breakthrough speeds were achieved without sacrificing precision and only on 8 sockets, showcasing the true capabilities of dataflow! Why would you buy 576 sockets and go to 8 bits when you can run using 16 bits and just 8 sockets. Try out the model and check out the speed here - coe-1.cloud.snova.ai.
📢New research on mechanistic architecture design and scaling laws.
- We perform the largest scaling laws analysis (500+ models, up to 7B) of beyond Transformer architectures to date
- For the first time, we show that architecture performance on a set of isolated token manipulation tasks is correlated with metrics of interest at scale, such as compute-optimal loss. Say hello to fast architecture improvement!
- Striped architectures consistently outperform homogeneous architectures, as they benefit from specialization of each layer type to particular subtasks
Today we are thrilled to share that we’ve raised $106M in a new round led by @SalesforceVC with participation from @coatuemgmt and our existing investors.
Our vision is to rapidly bring innovations from research to production and to ultimately build the best platform we can for developers, startups, and enterprises to run generative AI applications built on open-source models at production scale.
Excited to announce our new speculative decoding method, Sequoia!
Sequoia scales speculative decoding to very large speculation budgets, is robust to different decoding configurations, and can adapt to different hardware.
Serve Llama2-70B on one RTX4090 with half-second/token latency!
📢 Announcing our new speculative decoding framework Sequoia ❗️❗️❗️ It can now serve Llama2-70B on one RTX4090 with half-second/token latency (exact❗️no approximation)
🤔Sounds slow as a sloth 🦥🦥🦥???
Fun fact😛: DeepSpeed -> 5.3s / token; 8 x A100: 25ms / token (costs 8 x $18,000 = $140,000+ but an RTX4090 is $1000+😉)
Excited to demonstrate Mamba's potential as the backbone of DNA language models! This significantly extends preliminary results from the original paper, and the release comes with pretrained models - one of the most common requests we've gotten :)
We are excited to present Caduceus: bi-directional DNA language model built on Mamba, with long range modeling that respects inherent symmetry of double helix DNA structure.
Caduceus is SoTA on several benchmarks, including identifying causal SNPs for gene expression.
🧵1/9
Hazy Research: Strip Mall AI Research Club reposted
We are excited to present Caduceus: bi-directional DNA language model built on Mamba, with long range modeling that respects inherent symmetry of double helix DNA structure.
Caduceus is SoTA on several benchmarks, including identifying causal SNPs for gene expression.
🧵1/9
Hazy Research: Strip Mall AI Research Club reposted
Today we're excited to introduce Devin, the first AI software engineer.
Devin is the new state-of-the-art on the SWE-Bench coding benchmark, has successfully passed practical engineering interviews from leading AI companies, and has even completed real jobs on Upwork.
Devin is an autonomous agent that solves engineering tasks through the use of its own shell, code editor, and web browser.
When evaluated on the SWE-Bench benchmark, which asks an AI to resolve GitHub issues found in real-world open-source projects, Devin correctly resolves 13.86% of the issues unassisted, far exceeding the previous state-of-the-art model performance of 1.96% unassisted and 4.80% assisted.
Check out what Devin can do in the thread below.
Hazy Research: Strip Mall AI Research Club reposted
Excited to release Based, an architecture that combines two✌️ simple, familiar, attention-like primitives – short (size-64) sliding window attention and softmax-approximating linear attention – to enable high quality and efficient inference! 💨 🚀 joint w/ @EyubogluSabri, @mzhangio, @HazyResearch and team!
Hazy Research: Strip Mall AI Research Club reposted
Releasing our Based preprint, code, initial models
Like others, we’ve found attention is still great. But 3 simple ideas to make it better:
☝️Too expensive? Use exact attn in small sliding windows
✌️Doesn’t capture long range? Fill in the gaps w expressive linear attentions (Taylor series approx)
👌Still not fast / high-thruput enough? Play on even footing and bring your own hardware-aware kernels to the table; *optimized* for tensor cores / the “future of GPUs”
Result?
We find these ideas 1. outperform other subquadratic models (Mamba, RWKV, Hyena);
2. close gap w Transformers on recall / ICL tasks; work across modalities (DNA?)
Excited to release Based, an architecture that combines two✌️ simple, familiar, attention-like primitives – short (size-64) sliding window attention and softmax-approximating linear attention – to enable high quality and efficient inference! 💨 🚀 joint w/ @EyubogluSabri, @mzhangio, @HazyResearch and team!
Hazy Research: Strip Mall AI Research Club reposted
In some new work (the first from the new lab!), we lay out a vision for a biological foundation model that unites DNA, RNA, and protein modalities and operates at molecular, systems, and genome levels of scale.
Introducing Evo: a long-context biological model based on StripedHyena that generalizes across DNA, RNA, and proteins. It is capable of prediction tasks and generative design, from molecular to whole genome scale (over 650k tokens in length). www.together.ai/blog/evo
Hazy Research: Strip Mall AI Research Club@HazyResearch
Hazy Research: Strip Mall AI Research Club@HazyResearch
Thank you so much for the fun keynote, @NeurIPSConf
As in every year, our lab had a blast! We've enjoyed connecting with so many smart, enthusiastic people--and learning about your work. What an exciting time in AI!
KV-cache got you down? Sharing Based✌️, a simple architecture built from PyTorch 101 building blocks (convolutions, linear attention). It gives exciting quality vs the modern architectures & its hidden state size is fixed, enabling 4.5x >throughput vs attn hazyresearch.stanford.edu/blog/2023-12-11-zoology2-based
Hazy Research: Strip Mall AI Research Club reposted
Curious whether sub-quadratic LMs like RWKV and Hyena will replace Transformers? We find that Transformers are still much better at associative recall (AR): a simple task known to be essential for in-context learning. hazyresearch.stanford.edu/blog/2023-12-11-zoology1-analysis
Check out the new Mamba model from Cartesia cofounder & CMU prof @_albertgu.
Mamba is the first state space architecture to outperform Transformers on language modeling at 1.5B/3B scale, using a new data-dependent *selection* mechanism.
Quadratic attention has been indispensable for information-dense modalities such as language... until now.
Announcing Mamba: a new SSM arch. that has linear-time scaling, ultra long context, and most importantly--outperforms Transformers everywhere we've tried.
Quadratic attention has been indispensable for information-dense modalities such as language... until now.
Announcing Mamba: a new SSM arch. that has linear-time scaling, ultra long context, and most importantly--outperforms Transformers everywhere we've tried.
Personal update: I recently joined the amazing team of @togethercompute as a Distinguished Research Scientist!
My primary focus will be on efficient training methods and decentralized DL, and we plan to have lots of open-source releases and publications — stay tuned for updates!
Hazy Research: Strip Mall AI Research Club reposted
Transformers power most advances in LLMs, but its core attention layer can’t scale to long context.
With @_albertgu, we’re releasing Mamba, an SSM architecture that matches/beats Transformers in language modeling, yet with linear scaling and 5x higher inference throughput. 1/
Quadratic attention has been indispensable for information-dense modalities such as language... until now.
Announcing Mamba: a new SSM arch. that has linear-time scaling, ultra long context, and most importantly--outperforms Transformers everywhere we've tried.
The key idea: map the FFT onto tensor cores using a Monarch decomposition -- which allows kernel fusion for long sequences, and uses fast matmul units to compute the FFT (pic 2).
The FFT convolution allows us to compute the convolution in O(N log N) time, instead of O(N^2) from computing it directly (as in PyTorch nn.Conv1d). With advances in gated convolutions & gated SSMs, this means that we have a sub-quadratic alternative to attention that scales well!
But the FFT convolution has a critical flaw for ML - it has low hardware utilization on GPUs. We're talking several times slower than FlashAttention/FlashAttention-v2 until you get to very long sequences.
There's two critical bottlenecks: the FFT convolution incurs a lot of expensive I/O to store intermediate outputs, and it doesn't use tensor cores (16x faster than general arithmetic on A100/H100)! Kernel fusion can partially address the I/O problems (32K & shorter), but you run into SRAM limits at long sequences.
FlashFFTConv addresses these drawbacks, and achieves speedup over FlashAttention-v2 at sequences as short as 2K.
The core idea is that a Monarch decomposition allows us to break up the FFT into smaller parts that can be computed using matrix-matrix multiply (pic 3).
This decomposition brings another benefit: Since the FFT is being split into smaller parts, you can avoid SRAM limitations even for long sequences -- we support sequences up to length 4 million.
Now, there's a good old-fashioned systems tradeoff with this Monarch decomposition. You can recurse on the decomposition to break the FFT down into more parts -- compute the FFT in 2 parts, vs 3 parts or 4 parts. Higher-order decompositions reduce your FLOPs, but require more I/O between intermediate steps. This introduces a natural tradeoff, which we model using a cost model that takes both compute time and I/O time into account (pic 4).
(There's another tradeoff for higher-order -- if your sequence is too short, you actually get to matmuls that are too small to fill up your tensor cores). Our cost model gives us a natural guide -- we can use a order-2 decomposition for sequences up to 4K, then order-3 for sequences up to 32K, and then order-4 for sequences up to 4M. Even longer sequences may require even higher-order breakdowns!
The upshot of all of this is end-to-end speedup and higher utilization. We see up to 4.4x speedup for long sequence models (where a bunch of the gain also comes from memory reduction).
With these gains, long convolutional models are now competitive with FlashAttention-v2 at sequences as short as 2K -- with more speedup for longer sequences.
FlashFFTConv is already being used to support several internal research projects, and we'll be releasing new long-sequence models trained with FlashFFTConv this week -- stay tuned for more!
With great collaborators @KumbongHermann, @exnx, and @HazyResearch. Hermann is applying to PhD programs this year -- he's a beast, you should hire him!
Shoutouts to @tri_dao, @BeidiChen for discussions on early versions of this work and always pushing the boundaries of ML & systems :)
The key idea: map the FFT onto tensor cores using a Monarch decomposition -- which allows kernel fusion for long sequences, and uses fast matmul units to compute the FFT (pic 2).
The FFT convolution allows us to compute the convolution in O(N log N) time, instead of O(N^2) from computing it directly (as in PyTorch nn.Conv1d). With advances in gated convolutions & gated SSMs, this means that we have a sub-quadratic alternative to attention that scales well!
But the FFT convolution has a critical flaw for ML - it has low hardware utilization on GPUs. We're talking several times slower than FlashAttention/FlashAttention-v2 until you get to very long sequences.
There's two critical bottlenecks: the FFT convolution incurs a lot of expensive I/O to store intermediate outputs, and it doesn't use tensor cores (16x faster than general arithmetic on A100/H100)! Kernel fusion can partially address the I/O problems (32K & shorter), but you run into SRAM limits at long sequences.
FlashFFTConv addresses these drawbacks, and achieves speedup over FlashAttention-v2 at sequences as short as 2K.
The core idea is that a Monarch decomposition allows us to break up the FFT into smaller parts that can be computed using matrix-matrix multiply (pic 3).
This decomposition brings another benefit: Since the FFT is being split into smaller parts, you can avoid SRAM limitations even for long sequences -- we support sequences up to length 4 million.
Now, there's a good old-fashioned systems tradeoff with this Monarch decomposition. You can recurse on the decomposition to break the FFT down into more parts -- compute the FFT in 2 parts, vs 3 parts or 4 parts. Higher-order decompositions reduce your FLOPs, but require more I/O between intermediate steps. This introduces a natural tradeoff, which we model using a cost model that takes both compute time and I/O time into account (pic 4).
(There's another tradeoff for higher-order -- if your sequence is too short, you actually get to matmuls that are too small to fill up your tensor cores). Our cost model gives us a natural guide -- we can use a order-2 decomposition for sequences up to 4K, then order-3 for sequences up to 32K, and then order-4 for sequences up to 4M. Even longer sequences may require even higher-order breakdowns!
The upshot of all of this is end-to-end speedup and higher utilization. We see up to 4.4x speedup for long sequence models (where a bunch of the gain also comes from memory reduction).
With these gains, long convolutional models are now competitive with FlashAttention-v2 at sequences as short as 2K -- with more speedup for longer sequences.
FlashFFTConv is already being used to support several internal research projects, and we'll be releasing new long-sequence models trained with FlashFFTConv this week -- stay tuned for more!
With great collaborators @KumbongHermann, @exnx, and @HazyResearch. Hermann is applying to PhD programs this year -- he's a beast, you should hire him!
Shoutouts to @tri_dao, @BeidiChen for discussions on early versions of this work and always pushing the boundaries of ML & systems :)
I'm very excited to share that I've started as an Assistant Professor of Computer Science at Stanford University! My lab will focus on self-improving AI methodologies and systems. Looking forward to working with the next generation of researchers! (azaliamirhoseini.com)
Hazy Research: Strip Mall AI Research Club reposted
We’re beyond excited to share the first release of LegalBench–a collaboratively constructed open-source benchmark for evaluating legal reasoning in English large language models.
Announcing FlexFlow Serve! A low-latency, high-performance distributed system for serving LLMs, reducing inference latency by 1.8-2.2x compared to existing systems. Easy to install: pip install flexflow Repo: github.com/flexflow/FlexFlow A few reasons you may want to try it. 1/n
Hazy Research: Strip Mall AI Research Club reposted
Stanford's Madrona project is an ECS-based game engine that runs 10,000s of environments in parallel on a single GPU, reducing training from hours to minutes.
Large language models (LMs) rely heavily on training data quality. How do we best select training data for good downstream model performance across tasks? Introducing 🍳Skill-It: a data-driven framework for understanding and training LMs!
Large language models (LMs) rely heavily on training data quality. How do we best select training data for good downstream model performance across tasks? Introducing 🍳Skill-It: a data-driven framework for understanding and training LMs!
A new sub-quadratic BERT that is 25% more parameter efficient, faster and can scale to long sequences. This is an exciting direction! together.ai/blog/monarch-mixer
Hazy Research: Strip Mall AI Research Club reposted
Excited to share our recent work towards sub-quadratic architectures that improve efficiency and may be more accessible in low-resource environments, while preserving Transformer-level quality!!! Releasing M2-BERT today 💎
Transformers are the workhorse architecture behind modern foundation models. But they are limited in their ability to capture knowledge & scale.
In partnership with @HazyResearch, we’re releasing a first look at Monarch Mixer—an exploration into a more efficient, sub-quadratic model architecture.
We’re thrilled to partner with Together AI to enable any enterprise to build proprietary LLMs on their data tailored to their specific needs. Learn how both data-centric and model-centric operations are needed to build GPT-You for your business. snkl.ai/tgth
Hazy Research: Strip Mall AI Research Club reposted
Announcing FlashAttention-2! We released FlashAttention a year ago, making attn 2-4 faster and is now widely used in most LLM libraries. Recently I’ve been working on the next version: 2x faster than v1, 5-9x vs standard attn, reaching 225 TFLOPs/s training speed on A100. 1/
Hazy Research: Strip Mall AI Research Club reposted
Announcing FlashAttention-2! We released FlashAttention a year ago, making attn 2-4 faster and is now widely used in most LLM libraries. Recently I’ve been working on the next version: 2x faster than v1, 5-9x vs standard attn, reaching 225 TFLOPs/s training speed on A100. 1/
Hazy Research: Strip Mall AI Research Club@HazyResearch
Very excited to announce that I've finished my PhD @Stanford and will be joining @Princeton CS department as an Assistant Professor in Fall 2024. Looking forward to working with students and colleagues @PrincetonCS on ML & systems!
Hazy Research: Strip Mall AI Research Club reposted
I'm delighted to announce that I'll be moving permanently to CMU @CSDatCMU starting this fall. I'm excited to team up with the amazingly talented @andy_pavlo. We hope many of you will come visit us in Pittsburgh.
Blow out news! @pateljm is joining me at @CMUDB this fall semester. We will hold a live press conference in August once Jignesh gets settled. CMU's DB squad is now primed to rival the Germans at TUM and the Dutch at CWI! Props to @sriniseshan for turning this one out!
I'm delighted to announce that I'll be moving permanently to CMU @CSDatCMU starting this fall. I'm excited to team up with the amazingly talented @andy_pavlo. We hope many of you will come visit us in Pittsburgh.
Do we need Attention? (v0 github.com/srush/do-we-need-attention/blob/main/DoWeNeedA…): Slides for a survey talk summarizing recent Linear RNN models with a focus on NLP. Tries to cover a lot of different S4-related models (as well as RWKV/MEGA) in a digestible way.
Hazy Research: Strip Mall AI Research Club reposted
ACM and @ComputerSociety have named Kunle Olukotun, a Professor @Stanford, as the recipient of the 2023 ACM-IEEE CS Eckert-Mauchly Award for contributions & leadership in the development of parallel systems. Please join us in congratulating the recipient! awards.acm.org/eckert-mauchly
Hazy Research: Strip Mall AI Research Club reposted
• Instruct, chat, base, and interim checkpoints on @huggingface • The instruct model outperforms all open 7B models on HELM benchmarks • The 5TB dataset has been used to train over 100 models
... or on AMD, Nvidia, Intel GPUs, Apple Silicon, iPhones, and Android phones. Excited by the possibilities this opens up for personal, private LLMs trained and running on your local device! #opensourceai#mlcllm
Introducing 100K Context Windows! We’ve expanded Claude’s context window to 100,000 tokens of text, corresponding to around 75K words. Submit hundreds of pages of materials for Claude to digest and analyze. Conversations with Claude can go on for hours or days.
Hazy Research: Strip Mall AI Research Club@HazyResearch
StarCoder is a 15B LLM for code with 8k context and trained only on permissive data in 80+ programming languages. It can be prompted to reach 40% pass@1 on HumanEval and act as a Tech Assistant.
Excited to introduce Distilling Step-by-Step! ⚗️🪜 A simple mechanism to train small task-specific models to outperform LLMs, by leveraging less data needed by standard finetuning/distillation.
Excited to be part of this project showing the power of #datacentricai for foundation models!
Tl/dr: hold the model architecture & training code fixed... iterate only on the data... get a new state-of-the-art score on CLIP (given fixed compute)!!
As part of building the Together Decentralized Cloud, we care deeply about the impact on the environment. That's why we are excited to share how Together and @CrusoeEnergy are reducing the carbon impact of generative AI.
Announcing RedPajama — a project to create leading, fully open-source large language models, beginning with the release of a 1.2 trillion token dataset that follows the LLaMA recipe, available today! www.together.xyz/blog/redpajama More in 🧵 …
Hazy Research: Strip Mall AI Research Club reposted
Excited to see the RedPajama dataset released: check out the @MeerkatML data exploration dashboard we put together in a collaboration with @togethercompute as part of this release 🚀
We’ll continue to update and add to that in the RedPajama repo!
Announcing RedPajama — a project to create leading, fully open-source large language models, beginning with the release of a 1.2 trillion token dataset that follows the LLaMA recipe, available today! www.together.xyz/blog/redpajama More in 🧵 …
Hazy Research: Strip Mall AI Research Club reposted
1/ Why do some of us use the term "foundation model" (FM) instead of LLM/GenAI? - Not just language- any data type - Not just generative use cases- all AI application types (e.g. predictive/discriminative) - Foundation models are foundations- still need to build the house on top!
Hazy Research: Strip Mall AI Research Club reposted
Announcing RedPajama — a project to create leading, fully open-source large language models, beginning with the release of a 1.2 trillion token dataset that follows the LLaMA recipe, available today! www.together.xyz/blog/redpajama More in 🧵 …
Hazy Research: Strip Mall AI Research Club reposted
Text is great. It gives us tikz unicorns. But embeddings are still critical for retrieval, scoring, and probing. Manifest (github.com/HazyResearch/manifest) now supports both! Use Manifest for getting embeddings with OpenAI and local models from HF.
Hazy Research: Strip Mall AI Research Club reposted
Excited to speak next Thurs at the LLMs in prod conference @mlopscommunity
LLMs are becoming the "foundations" for all AI dev. But they're not an out-of-the-box solution for most real problems. I'll share how data-centric AI is the key to "building the house" on top!
Hazy Research: Strip Mall AI Research Club reposted
Foundation models are amazing for human-in-the-loop tasks. But, we're also excited to apply them to batch computing tasks where we can't check every output and require high throughput.
Foundation models have enabled amazing human-in-the-loop systems (ChatGPT, Copilot, ++). How can we bring them to bear on important batch computing tasks (like information extraction) - where we need efficiency and reliability at scale?
In this 2min video, @SnorkelAI cofounder Brendan Hancock explains how they unlocked foundation models for their customers using 🤗 Inference Endpoints 🍿+🧵
Have you ever wanted to do an experiment on LLMs and found that none of the existing model suites met your needs? At @AiEleuther we got tired of this happening and so designed a model suite that centers enabling scientific research as its primary goal
Ramp just turned 4 years old. I’m proud to share that in ’22, @tryramp’s revenue grew an industry leading 4X YoY. By obsessing over the value we deliver for customers we entered '23 as the industry’s fastest growing finance automation platform. Here’s how: www.ramp.com/blog/future-of-finance-running-on-ramp
Hazy Research: Strip Mall AI Research Club reposted
FlashAttention is now in PyTorch 2.0, making Transformers faster and much more memory-efficient especially for long sequences 🚀. Thanks to the PyTorch devs for this integration!
We're super excited to see how much farther we can take these models - with linear scaling, can we get to a million tokens? A billion? Come along with us to find out!
Hazy Research: Strip Mall AI Research Club reposted
LLMs today are amazing, but how to get time series foundation models?
@ICLR2023 we share some ideas w SpaceTime🌌 ➡️New architecture for SoTA forecasting, classification ➡️Comes w expressive modeling; fast + flexible decoding; long-context
Key ideas, paper, code, fun demo👇
Hazy Research: Strip Mall AI Research Club reposted
Attention is great. Are there other operators that scale?
Excited to share our work on Hyena, an alternative to attn that can learn on sequences *10x longer*, up to *100x faster* than optimized attn, by using implicit long convolutions & gating
Today we release LLaMA, 4 foundation models ranging from 7B to 65B parameters. LLaMA-13B outperforms OPT and GPT-3 175B on most benchmarks. LLaMA-65B is competitive with Chinchilla 70B and PaLM 540B. The weights for all models are open and available at research.facebook.com/publications/llama-open-and-efficie… 1/n
Hazy Research: Strip Mall AI Research Club reposted
IMO: Models like ChatGPT *are* the future- but will have to integrate more of the "past" to actually work (eg. retrieval modules + task-specific fine-tuning).
Hazy Research: Strip Mall AI Research Club reposted
What if zero-shot generalization was directly induced by explicit multitask learning? @ajratner & @stevebach discuss how @BigscienceW researchers trained models to attain strong ZSL perf on several datasets, outperforming models up to 16x their size. www.youtube.com/watch?v=BiErdldKRPI
Hazy Research: Strip Mall AI Research Club reposted
We’re excited to announce the close of our $35M Series A financing! Atomic AI is fusing artificial intelligence and structural biology to unlock RNA drug discovery, designing RNA-targeted and RNA-based medicines to treat undruggable diseases. www.businesswire.com/news/home/20230125005256/en/Atomic-A…
Hazy Research: Strip Mall AI Research Club reposted
Introducing FlashConv, a new technique for training state space models. Runs up to 35X faster than FlashAttention and runs the new H3 language model 2.4X faster than Transformers! Research by @tri_dao and our own @realDanFu. www.together.xyz/blog/h3
Hazy Research: Strip Mall AI Research Club reposted
Attention is all you need... but how much of it do you need?
Announcing H3 - a new generative language models that outperforms GPT-Neo-2.7B with only *2* attention layers! Accepted as a *spotlight* at #iclr2023! 📣 w/ @tri_dao
Excited to announce a few updates in our work towards using foundation models for data wrangling tasks!
📜Our paper has been published in VLDB 2023 💪Added 2 new integration tasks: data transformation & schema matching ✍️We wrote a blog post 🏋Added data tasks to HELM benchmark
Hazy Research: Strip Mall AI Research Club reposted
2022 was a big year for team Snorkel: we tripled our customer base, introduced programmatic labeling and foundational model innovations, and significantly expanded integrations across the ML stack -- accelerating AI development by 10-100x. 📈www.businesswire.com/news/home/20230118005384/en/Snorkel-…
Hazy Research: Strip Mall AI Research Club reposted
At 11:45 AM Pacific, @simran_s_arora, ML researcher @HazyResearch, will dive deeper into how a 6B parameter open-sourced model performed better than 175B parameter GPT-3 on 15 tasks. Check out other talks and sign up here: snorkel.ai/events/
Hazy Research: Strip Mall AI Research Club reposted
I’ve been working with @AdeptAILabs and we’ve made FlashAttention even faster for long sequences! For seqlen 8K, FlashAttention is now up to 2.7x faster than a standard PyTorch implementation even at small batch, making it easier to train better LMs with longer context 1/7
Announcing FlashAttention, a fast and memory-efficient attention algorithm with no approximation! 📣 w/ @realDanFu
By reducing GPU memory reads/writes, FlashAttention runs 2-4x faster & requires 5-20x less memory than PyTorch standard attention, & scales to seq. length 64K. 1/
Hazy Research: Strip Mall AI Research Club reposted
Ce Zhang (@DS3Lab and @togethercompute) has done some crazy stuff in distributed training. In this talk, he goes over the magic behind distribute training and inference on a GLOBAL scale over slow networks!
I’ve been working with @AdeptAILabs and we’ve made FlashAttention even faster for long sequences! For seqlen 8K, FlashAttention is now up to 2.7x faster than a standard PyTorch implementation even at small batch, making it easier to train better LMs with longer context 1/7
Announcing FlashAttention, a fast and memory-efficient attention algorithm with no approximation! 📣 w/ @realDanFu
By reducing GPU memory reads/writes, FlashAttention runs 2-4x faster & requires 5-20x less memory than PyTorch standard attention, & scales to seq. length 64K. 1/
Hazy Research: Strip Mall AI Research Club reposted
This quarter, Stanford’s Advances in Foundation Models Class (CS 324) will be partnering with the Stanford MLSys Seminar to host a special talk series on foundation models!
1/ ‘22 was a big year for @SnorkelAI! We - Introduced new foundation model & data-centric AI paradigms - More than tripled our customer base, incl. 5/10 top US banks + top-10 in 7 other industries, accelerating their AI dev by 10-100x+ - Welcomed 40 new Snorkelers
And much more!
Hazy Research: Strip Mall AI Research Club reposted
It’s hard work to make evaluations for language models (LMs). We’ve developed an automated way to generate evaluations with LMs, significantly reducing the effort involved. We test LMs using >150 LM-written evaluations, uncovering novel LM behaviors. www.anthropic.com/model-written-evals.pdf
Hazy Research: Strip Mall AI Research Club reposted
Details: We took Hugging Face’s Transformer implementation, added FlashAttention, built our own tokenizer, and trained over 300B tokens (110 GB text) on 128 A100 GPUs for ~6.25 days. We did full fine-tuning on downstream tasks (e.g., MedQA-USMLE) for evaluation.
Hazy Research: Strip Mall AI Research Club reposted
📣 CRFM announces PubMedGPT, a new 2.7B language model that achieves a new SOTA on the US medical licensing exam. The recipe is simple: a standard Transformer trained from scratch on PubMed (from The Pile) using @mosaicml on the MosaicML Cloud, then fine-tuned for the QA task.
Hazy Research: Strip Mall AI Research Club reposted
Let’s shed new light on classical algorithms, make ML more accessible to the general community, and advance interdisciplinary research (science?!) together!
🙏Plz help spread the world.
Hazy Research: Strip Mall AI Research Club reposted
It's all about the data + supervision! Whether creative fine-tuning (eg. FLAN), careful training data curation (eg. StableDiffusion, Whisper, etc.), RLHF- it all comes back to data-centric development.
Data dev + flywheels will be the big differentiators in this space.
Exciting but overlooked that ChatGPT is primarily an alignment advance—the base model (GPT-3.5) has been available in publicly for many months, but making it into a useful chat system required significant strides with reliably following the intent of the developer and the user.
Hazy Research: Strip Mall AI Research Club reposted
It might make mistakes in reasoning and knowledge retrieval, this is not worth overindexing on in my opinion. This is certain to improve quickly, although it’s good to know what’s not working quite as well yet
Hazy Research: Strip Mall AI Research Club reposted
🧵 A longish thread on big models, ChatGPT, and emergence of non-trivial intelligence from big data & compute. 1st, please, do not misinterpret some of my tweets, @OpenAI made another amazing breakthrough and pushed the boundaries of conversational AI.
Hazy Research: Strip Mall AI Research Club reposted
Join us today Wed 30 at J#431 for fractals and collage representations (arxiv.org/abs/2204.07673) and J#123 for learning in frequency domain, neural operators and long-range dependencies (arxiv.org/abs/2211.14453). Both at 11.00am - 1.00pm, catch me bouncing between posters!
Hazy Research: Strip Mall AI Research Club reposted
Try out GPT-JT on Hugging Face and let us know your feedback! The app uses our decentralized cloud to provide super fast responses. We would love to work together with you on v2!
If you're at #neurips2022, @togethercompute will present two papers on decentralized learning, including one oral, with solutions to the communication cost under slower networks. Come by today’s poster session [11:00 AM CST, Hall J #621]! We are excited to share AC-SGD (1 of 3):
Hazy Research: Strip Mall AI Research Club reposted
We're releasing an optimized implementation of GPT2/GPT3 with FlashAttention🚀! This trains 3-5x faster than the Huggingface version, reaching up to 189 TFLOPs/sec per A100, 60.6% (model) FLOPs util of the theoretical maximum. 1/6 github.com/HazyResearch/flash-attention/tree/main/training
Hazy Research: Strip Mall AI Research Club reposted
Language models are becoming the foundation of language technologies, but when do they work or don’t work? In a new CRFM paper, we propose Holistic Evaluation of Language Models (HELM), a framework to increase the transparency of LMs. Holistic evaluation includes three elements:
Hazy Research: Strip Mall AI Research Club reposted
Amazing, H100 is 2.6x faster than A100 in MLPerf training 2.1! And FlashAttention makes the super-optimized BERT implementation from Nvidia go even faster. Congrats to the Nvidia team for such great work! developer.nvidia.com/blog/leading-mlperf-training-2-1-wit…
Hazy Research: Strip Mall AI Research Club reposted
Upon request from @davisblalock, I can share some thoughts here. Overall, this paper provides a great deep dive into long-range CNNs that can hopefully increase excitement about some overlooked models! With that said, here are some subtleties and nits that could be pointed out:
Hazy Research: Strip Mall AI Research Club reposted
1/ On properties: The SSM line of work is explicit that they *are* just convolutions, which is a connection I often encourage for intuition. But the amazing thing is that they're *also* RNNs (as well as continuous-time), giving them many more properties beyond the conv. view
Hazy Research: Strip Mall AI Research Club reposted
An absolute pleasure working with the brilliant @jacopotagliabue on a blog post / code repo integrating Large Language Models into our Modern Data Stack workflows. Check it out👇🏾!
We've @hazyresearch been thinking a lot about foundation models and if we're entering the "data centric" era of FMs! Here are some of our thoughts around what we can learn from the past data-centric revolutions and what is really exciting and new with FMs: hazyresearch.stanford.edu/blog/2022-10-11-datacentric-fms
Hazy Research: Strip Mall AI Research Club reposted
We spent a couple days this week speeding up Stable Diffusion in @huggingface Diffusers using FlashAttention. 3-4x faster than the original version, 33% faster than the super optimized v0.4.1 - and >1 image/s throughput on A100. w/ @tri_dao
A short thread on how we did it👇
Hazy Research: Strip Mall AI Research Club reposted
So honored that our Monarch paper was selected as an Outstanding paper at ICML! This type of matrices can bring wall-clock time speedup with sparse training while maintaining model quality.
@tri_dao will present our work #icml2022 Monarch: Expressive Structured Matrices for Efficient and Accurate Training at Ballroom #1 at 2pm! Come and join us in our poster session today as well. Super thrilled that we won an *outstanding paper award*!!! 🚀
Hazy Research: Strip Mall AI Research Club reposted
Super excited our work on label-efficient, robust MRI reconstruction received best paper award @midl_conference! thanks to the organizers for such an awesome conference!
Best paper award goes to VORTEX: Physics-Driven Data Augmentations Using Consistency Training for Robust Accelerated MRI Reconstruction!
Hazy Research: Strip Mall AI Research Club@HazyResearch
"Performing MLPerf benchmarks is no easy task, and often involves the work of many engineers. But a single graduate student, with some consultation, can do it, too. @tri_dao was that graduate student." spectrum.ieee.org/mlperf-rankings-2022
Tri on the market next year!!! HE IS AMAZING!
Hazy Research: Strip Mall AI Research Club reposted
New preprint with @russpoldrack and @HazyResearch in which we devise and evaluate self-supervised learning techniques for neuroimaging data based on prominent learning frameworks in NLP:
S4 is an amazing sequence model - but has seemed mysterious. It doesn't have to be!
In this blog (originally an internal explainer for our group), @HazyResearch looks at S4 from first principles that are familiar to most sophomore engineering students.
New survey paper! We discuss “emergent abilities” of large language models.
Emergent abilities are only present in sufficiently large models, and thus they would not have been predicted simply by extrapolating the scaling curve from smaller models.
In this blog post, @HazyResearch, @tri_dao, @realDanFu, and @krandiash explain why their answer is *longer sequences*, and highlight recent work in this direction (FlashAttention, S4). Check it out!
Do you ever wonder if your model - despite logging impressive accuracy - is still failing on an important but unknown slice of your dataset? We certainly do! Stoked to share recent work @iclr22 in which we develop & evaluate ~slice discovery methods~ (1/7) ai.stanford.edu/blog/domino/
Hazy Research: Strip Mall AI Research Club reposted
Do you ever wonder if your model - despite logging impressive accuracy - is still failing on an important but unknown slice of your dataset? We certainly do! Stoked to share recent work @iclr22 in which we develop & evaluate ~slice discovery methods~ (1/7) ai.stanford.edu/blog/domino/
Hazy Research: Strip Mall AI Research Club reposted
Today, @tryramp is announcing a new round at an $8.1 billion valuation. It crossed $100 million in recurring revenue faster than any SaaS or fintech company in history.
That's just the first act. It has a Google-sized opportunity ahead of it.
Reasoning over both public and *private* data is necessary for personalized ML systems. But how can we use personal context without exposing it to the world? 🔑🔒 We explore this question in new work on personalized and private retrieval systems!! dl.fbaipublicfiles.com/concurrentqa/reasoning_over_public…
Building systems that can securely reason over private data to answer questions or reason about the world is challenging & important. Meta is introducing a new data set, ConcurrentQA, and a new privacy methodology to encourage research in these areas.ai.facebook.com/blog/building-systems-to-reason-securely-…
Hazy Research: Strip Mall AI Research Club reposted
Excited to share work improving ML robustness to spurious correlations! arxiv.org/abs/2203.01517
We propose Correct-N-Contrast, a contrastive learning method that trains models to ignore spurious correlations while learning class-separable representations.
Deets in 🧵👇 (1/9)
Hazy Research: Strip Mall AI Research Club reposted
Our paper on Declarative Machine Learning Systems, covering bot Overton and Ludwig, is now on Communications of ACM. Hope it will be an interesting read. @HazyResearch
ResNet not working? Use our #neurips2021 paper to find what to use in-place of convs by searching our space of “XD-operations” containing convs, Fourier neural operators, graph convs, SOTA ops for neural PDE solvers, and infinitely many more arxiv.org/abs/2103.15798 [1/n]
Hazy Research: Strip Mall AI Research Club reposted
Deep gratitude for @jack and our entire team, and so much excitement for the future. Here’s the note I sent to the company. Thank you all for your trust and support 💙
Super exited to announce 🍵SKM-TEA🍵, a multi-task dataset designed for experimenting with the end-to-end medical imaging workflow. Check out our GitHub to get started⬇️⬇️⬇️ (1/3)
TLDR: New public 1.5TB medical imaging dataset for multi-tasking with raw k-space MRI data, morphological and quantitative T2 map images, along with segmentation, classification, and detection labels! Read on about Stanford Knee MRI with Multi-Task Evaluation (SKM-TEA)! 1/3
Join us at @StanfordHAI@ETH_AI_Center#datacentricai virtual Workshop to discuss this emerging topic. 2 Pacific mornings + 1 Zurich afternoon: a great line-up of speakers & a Policy+AI panel w/ policymaker @CantonZurich+legal+tech experts!
(1/n) Excited to release 2 preprints that describe our progress on sequence modeling for long-range dependencies! arxiv.org/abs/2110.13985 (NeurIPS ‘21) arxiv.org/abs/2111.00396 We build a new class of state space models that improve perf. on the Long Range Arena by 20 points!
Hazy Research: Strip Mall AI Research Club reposted
A thousand thanks to the amazing xView3 team from @DIU_x, @GlobalFishWatch, @USCG, @NOAA, & NMIO. We all think that solving big problems requires collaboration between governments, NGOs, industry, & academia alike, and -- with your help -- hope to do just that with xView3! (6/6)
Hazy Research: Strip Mall AI Research Club reposted
Excited that xView3 -- the ML challenge I've been working on for the last year that focuses on building ML models to detect illegal, unregulated, and unreported (IUU) fishing on synthetic aperture radar (SAR) -- just went live.
AlphaFold for RNA? Excited to share our @StanfordAILab work on deep learning for predicting 3D RNA structure, out today on the cover of @ScienceMagazine!
Excited for this discussion! Data centric AI is not just a technical idea- it’s a community focus one. Data is not just “preprocessing” for AI dev- it’s the heart and center. Excited to discuss @SnorkelAI ‘s unique programmatic approach + the data centric shift more broadly!
Today we’re launching AI21 Studio, a platform where you can instantly access our state-of-the-art language models to build your own applications - including a 178B parameters model, Jurassic-1 Jumbo. We can’t wait to see what you create! www.ai21.com/blog/announcing-ai21-studio-and-jurassic-1
Hazy Research: Strip Mall AI Research Club reposted
You can now get a broad overview of Data Centric AI there -- we've got discussion on weak supervision, self supervision, robustness, data augmentation, privacy, data selection, and more.
Check it out!!
Hazy Research: Strip Mall AI Research Club reposted
We need to move beyond manual #datalabeling for AI to live up to the hype. Make training data creation and management part of the development process with Snorkel Flow, the first #datacentricai platform powered by a programmatic approach ↓ snkl.ai/pti
Hazy Research: Strip Mall AI Research Club reposted
In nervous anticipation of my #acl2021nlp keynote, I recorded myself giving my talk, and I've posted that version for people who can't attend the live event. The video has high-quality captions, and you don't particularly need video to follow along: youtu.be/t_A36DDcG_0
Hazy Research: Strip Mall AI Research Club reposted
Do you know there's more than one Michael Jordan?🤨 Of course you do. Do retrievers?🤔 Our #acl2021nlp paper evaluates the entity disambiguation capabilities of retrievers & how popularity bias affects retrieval
Idea: Construct sets of queries for all entities which share a name
Hazy Research: Strip Mall AI Research Club reposted
ICML 2021 (4) Outstanding Paper Award Honorable Mentions: 1/4 Yucheng Lu and Christopher De Sa 📜Optimal Complexity in Decentralized Training (Tuesday 8am US Eastern)
Hazy Research: Strip Mall AI Research Club reposted
Hello World! We are delighted to announce that Opaque Systems has raised $9.5 million in Series Seed funding from @intelcapital , @RaceCapital , @thehousefund and FactoryHQ. Check out today's article from TechCrunch for more details on the news. tcrn.ch/3xpIXvJ
Hazy Research: Strip Mall AI Research Club reposted
Virginia Smith (assistant professor @mldcmu), and Priya Donti (Ph.D. candidate @CSDatCMU and Engineering and Public Policy departments), have been named to @techreview annual list of Innovators Under 35.
Excited to release Meerkat, a new data library for interactive machine learning! We've (@jundesai, @EyubogluSabri, @HazyResearch) been building this up over the last couple of months.
Announcing a new release of @ludwig_ai! It's the best yet, runs distributed very easily thanks to @raydistributed, reads file from remote FS, has improved hyperopt, great performance on tabular data (thanks @sercanarik), and new datasets, encoders and combiners. Check it out!
After 7yrs of research, @danavanaken + I are announcing that the commercial version of OtterTune is now available. This is a new easy-to-use SaaS for automatically optimizing @PostgreSQL + @MySQL configurations. See our inaugural blog post for more info: ottertune.com/blog/lets-do-this
Hazy Research: Strip Mall AI Research Club reposted
Just how much does domain-specific pretraining help for legal NLP tasks? We studied this by creating “CaseHOLD” -- a new benchmark for precedential reasoning in law.
How you/we can help --> a thread of threads on useful (live-updating) info/awareness/resources/oxygen/donating/volunteering/calling/service... #covidindiahelp
Hazy Research: Strip Mall AI Research Club reposted
News! cleanlab 1.0 launches today! cleanlab is the standard package for data-centric machine learning with noisy labels and fixing label errors in datasets. github.com/cgnorthcutt/cleanlab/releases/tag/v1.0
Hazy Research: Strip Mall AI Research Club reposted
New Relic is once again named a Leader in the 2021 @Gartner_inc Magic Quadrant for Application Performance Monitoring! We truly couldn’t have achieved this recognition without the help of our community.
The International Conference on Artificial Intelligence and Statistics (AISTATS) 2021 is being hosted virtually from April 13th - April 15th. We’re excited to share all the work from SAIL that’s being presented in this blog post: ai.stanford.edu/blog/aistats-2021/
Hazy Research: Strip Mall AI Research Club reposted
Super excited to partner with Softbank, GIC, and Temasek as new investors at SambaNova! Thanks to BlackRock, GV, Intel Capital, Walden, WRV, and Nepenthe Capital for participating in the round! www.usnews.com/news/technology/articles/2021-04-13/ai-chi…
Hazy Research: Strip Mall AI Research Club reposted
.@tryramp, developer of a corporate card to help businesses spend less, reached unicorn status after raising $115 million in a Series B led by D1 Capital Partners and @stripe. @ChristineMHall talked to co-founder @eglyman about plans for the new funding.
DS3Lab Blog #3: How should a user in the future interact with ML platforms? How much signals can we extract from cognitive features from a user (EEG/Eye-tracking) and boost the quality of ML models "for free"? @norahollenstein started from these questions four years ago... 1/3
Hazy Research: Strip Mall AI Research Club reposted
Another Snorkel-driven @NatureComms paper published this week, led by Stanford Researcher and Snorkel Research member @jasonafries: weakly supervised #ner, combining large medical ontologies for state-of-the-art performance and rapid COVID-19 research! rdcu.be/chTpg
Hazy Research: Strip Mall AI Research Club reposted
#sigmod2021 Serverless/FaaS is playing an important role in analytics, while its role to ML training is not yet clear. Jiawei & @ShaoduoGan (w/ @stub_AS@anaklimovic & Gustavo helping us on FaaS) create LambdaML to depict FaaS/IaaS tradeoff & understand alternative FaaS designs
Hazy Research: Strip Mall AI Research Club reposted
👁️👁️ Do multilingual language models accurately predict human reading behavior? 👁️👁️ Together with @lisabeinborn and Lena Jäger from @cl_uzh we ask this question in our paper, which has just been accepted at #naacl2021. Stay tuned for the preprint!
Hazy Research: Strip Mall AI Research Club reposted
Here's the second part of our sentiment analysis with Ludwig series! here we fineture a BERT model (imported from @huggingface Transformers) on SST. Enjoy!
A thank you to all the amazing speakers we have over the months and to all the people who joined our livestreams or watched the videos and decided to subscribe. It’s very humbling
Hazy Research: Strip Mall AI Research Club reposted
We are honored by Snorkel's selection in the #enterprisetech30 List, a list of the top 30 most promising companies voted by 100+ leading VCs. Excited to propel the data-first revolution for enterprise AI. Thank you, @Wing_VC. www.enterprisetech30.com
Hazy Research: Strip Mall AI Research Club reposted
#aistats2021 w/ N ML models (eg trained over diff. data slices), how to pick the best one given fresh unlabeled production data? We abstract this q from many MLOps apps. Mohammad (w/ @arkrause) & @nmervegurel builds ModelPicker to tackle this problem arxiv.org/abs/2010.09818
Hazy Research: Strip Mall AI Research Club reposted
We've updated our survey paper on arxiv! Some of the updates include an extended section on hyperbolic embeddings, a description of DGI and a better distinction between local and global filtering.
🚀Excited to release Robustness Gym, a new Python evaluation toolkit for evaluating the robustness of NLP models, as part of a collaboration between Stanford, Salesforce Research and UNC Chapel-Hill.
We support our @StanfordAILab alum @timnitGebru, her critical research on fairness & ethics in AI, and her tireless work in organizing for a more ethical & inclusive AI community. It’s urgent that voices like hers be heard and that we prioritize work on inclusion & ethics in AI.
Hazy Research: Strip Mall AI Research Club reposted
My 2nd PhD student @danavanaken passed her defense today over Zoom! Dana is a leading expert on using ML to optimize database configs. Thnx to committee @MikeCafarella + @dave_andersen + Geoff Gordon
I would say that you should hire Dana but she's going to be CTO of @OtterTuneAI
Hazy Research: Strip Mall AI Research Club reposted
We're excited to announce WILDS, a benchmark of in-the-wild distribution shifts with 7 datasets across diverse data modalities and real-world applications.
I'm excited to announce that we've formed Ursa Computing ursacomputing.com, to continue the Ursa Labs mission for Apache Arrow-powered data science tools and to build new products to make enterprise data work faster and more efficient.
Kudos to @WillKnight for putting the organizers on the spot for the influence of corporate money on the NeurIPS conference. It's interesting to sit here and listen to the answers.
Hazy Research: Strip Mall AI Research Club reposted
We hope that our work will be a useful first step towards addressing hidden stratification. To learn more, check out the links above or come chat with us at NeurIPS!
#aaai2021 Many information extraction project gets stuck not in "extraction" but in getting documents into a machine readable form. In collab w/ @stfeuerriegel, @johannesmrausch developed DocParser, a PDF->JSON tool weakly trained over arXiv source codes arxiv.org/abs/1911.01702
Hazy Research: Strip Mall AI Research Club reposted
In our new @StanfordAILab blog post, @ShikharMurty and I discuss the problem of training deep learning models with natural language explanations for tasks in vision, NLP, and beyond:
Hazy Research: Strip Mall AI Research Club reposted
Srijan Kumar - Lighthouz AI (YC S24) 🤖@srijankedia
Please RT! I am looking for PhD students to join my research group at @GeorgiaTech. My group works in cutting-edge topics in data mining, ML, graphs, recsys, adversarial learning. We focus on applications in social media, public health, and security. Please get in touch!
Hazy Research: Strip Mall AI Research Club reposted
Wouldn't it be great if AI could do debug our programs for us?
Check out our latest blog post on using ML to repair programs from error messages, with a promising approach that leverages program-feedback graphs and self-supervised learning.
Thrilled to announce that I will join @SnorkelAI today!! Their open source Snorkel project is what I have been contributing to and passionate about over the last a few years. Now so excited and happy to continue working on the Snorkel ecosystem as part of the team!!
Hazy Research: Strip Mall AI Research Club reposted
Interested in data and building bridges with other disciplines? Check out this opportunity to join @BrownCSDept and @Brown_DSI. Open rank, and we're particularly interested in candidates working on real-world societal challenges such as social equity and justice, and health.
Welcome to new faculty member @SharonYixuanLi! Li is an assistant professor in deep learning and artificial intelligence. Born in a Chinese city famous for its beef noodles, she was surprised to find a restaurant in Madison offering the same kind! @uwcdis
I will be giving a talk at Women in Computer Science (WiSC) at Stanford next Wednesday. I will talk about research on open-world machine learning and will stick around for a casual Q&A at the end. Look forward to this and thanks @StanfordWiCS for organizing!
Hazy Research: Strip Mall AI Research Club reposted
A small amount of guidance can go a long way — BEE: ✅ explores relevant objects much more frequently than prior approaches ✅ works well on a real robot from images ✅ enables better offline RL on downstream tasks (7/8)
Hazy Research: Strip Mall AI Research Club reposted
(1/5) Excited to release Bootleg hazyresearch.stanford.edu/bootleg/, a SotA named entity disambiguation system that tackles the long tail of entities that appear infrequently in training. Bootleg achieves SotA on 3 NED benchmarks and outperforms a BERT-based baseline by 40 F1 over the tail.
Hazy Research: Strip Mall AI Research Club reposted
Congratulations to Christos Faloutsos @CSDatCMU, recipient of the Fredkin Professorship in Artificial Intelligence. His interests include large-scale data mining with an emphasis on graphs and time sequences, anomaly detection, tensors, and fractals. www.cs.cmu.edu/news/scs-celebrates-new-professorships
Hazy Research: Strip Mall AI Research Club reposted
Tune in this Thursday 3-4pm for the second seminar of the Stanford MLSys Seminar Series. @matei_zaharia will talk about lessons from building MLflow at Databricks.
#cidr2021 Excited that the EaseML overview paper is accepted at CIDR and cannot wait to talk about what we have been building over the last four years. While we are working hard on the final version & system demo, an overview is at ds3lab.inf.ethz.ch/easeml.html and this thread:
Hazy Research: Strip Mall AI Research Club reposted
Thank you! I’m deeply humbled. &thank all of my collaborators&students, especially Prof. Arnold Milstein @StanfordMed for the long term collaboration, mentorship & friendship. Truly hope #ai can do a lot more to help our clinicians and patients by bettering healthcare delivery.
For this week's @stanfordnlp seminar Thursday 10am PT, excited to have Yoon Kim (MIT-IBM Watson/incoming MIT EECS prof) speaking on Deep Unsupervised Learning of Syntactic Structure. Open to the public - non-Stanford affiliates register at nlp.stanford.edu/seminar/
Hazy Research: Strip Mall AI Research Club reposted
#pdftotree v0.5.0 just released from @HazyResearch, once abandoned but revived with a new scope: a converter tool from #pdf to #hocr with text, tables, and figures being recognized and preserved. It was developed primarily for #fonduer, but should now be more general-purpose.
Hazy Research: Strip Mall AI Research Club@HazyResearch
Hazy Research: Strip Mall AI Research Club@HazyResearch
Tune in Thursday 15 3pm PST for the first episode featuring Marco Tulio Ribeiro (@marcotcr) from Microsoft Research who will tell us about how to find pitfalls in production models with behavioral testing using checklists (Best paper ACL 2020). www.youtube.com/channel/UCzz6ructab1U44QPI3HpZEQ
Hazy Research: Strip Mall AI Research Club@HazyResearch
Happy to share our @emnlp2020 findings paper. Our method generates text conditionally retaining the diversity of the training set while robustly conveying content. arxiv.org/abs/2009.12046 [1/4]
Hazy Research: Strip Mall AI Research Club reposted
Hello Fall 2020! 🙌 Excited to teach my new grad course "Data Systems for ML" covering the whole ML lifecycle from sourcing data to scalable model building to ML deployment and more.
1/n We believe that model hubs (TFHub, Pytorch Hub) provide an amazing opportunity for ML usability by reusing ML modules. Supported by a multi-year Google Focused Award, we explore how to manage an online model hub in real-world. This leads to a series of exciting work & collab:
Hazy Research: Strip Mall AI Research Club reposted
Our third ML seminar @utexasece will be delivered by Virginia Smith (@gingsmith) from @mldcmu, tomorrow at 11am (CDT). Virginia will talk about "Heterogeneity in Federated Settings".
A pro-Cornyn Super PAC is using a photo of my tattoos to make me seem "radical." That's pretty funny to me.
You think I'm ashamed of them? They cover my shrapnel wounds from when my helicopter was shot down. They're a mark of my service to our country. I'm damn proud of them.
Hazy Research: Strip Mall AI Research Club reposted
Researchers at Microsoft Semantic Machines are taking a new approach to conversational AI—modeling dialogues with compositional dataflow graphs. Learn how the framework supports flexible, open-ended conversations, and explore the dataset and leaderboard: aka.ms/AA9oxf3
Hazy Research: Strip Mall AI Research Club reposted
My student @danish037 is taking time each week to help undergrad & MS students by answering questions about research (& I imagine also planning for grad school).
I am planning to reserve some time every week this semester to help undergraduate/masters students by answering any questions they might have around pursuing research.
Drawing from her experiences at #netflix, #nvidia, and @SnorkelML, @chipro will talk about bridging the gap between research and production for ML at @Ai4Conferences. The talk is online and Ai4 has free passes. Join us today at 2.50 PM EST!
Hazy Research: Strip Mall AI Research Club reposted
A few years after I completed SLE, when I was 21 or so, they asked me to give a speech at the SLE banquet, and I thought I'd post that as a tribute to Mancall and what he built. Later, I adapted this into my essay for graduate fellowships, and I believe it still.
Hazy Research: Strip Mall AI Research Club reposted
A few years after I completed SLE, when I was 21 or so, they asked me to give a speech at the SLE banquet, and I thought I'd post that as a tribute to Mancall and what he built. Later, I adapted this into my essay for graduate fellowships, and I believe it still.
Hazy Research: Strip Mall AI Research Club reposted
Excited to share that @afreshai raised another $12M to advance our mission of using AI to reduce waste across the food supply chain. We're always looking for talented folks who are passionate about AI, food, and the environment to join our growing team. techstartups.com/2020/07/20/afresh-bags-12-million-series…
Hazy Research: Strip Mall AI Research Club reposted
💣 Drop: After 6 years of research at CMU, I am spinning out the OtterTune project as a start-up with my students @danavanaken + @bohanzhang14: ottertune.com
Not only has Professor Miron Livny been awarded the @IEEEorg Award for Outstanding Technical Achievement, but he has also now been awarded an IEEE TDCP ICDCS High Impact Paper Award! Congratulations! @Morgridge_Inst@DiscoveryBldg@UWMadisonLS
Looking for emerging AI talent? Join AI4ALL's Virtual Internship and Research Fair! This is a free opportunity for research labs and industry organizations to connect with rising underrepresented talent in AI. Apply here by September 1st: www.tfaforms.com/4833868
Hazy Research: Strip Mall AI Research Club reposted
I am saddened to write the passing of Dr. Sam Gambhir. He was my PhD advisor, friend, and mentor of 20 years. It's a tremendous loss for me, Sam's family, @Stanford, and science community. I will miss him dearly. #rip @StanfordCVI@StanfordMed@StanfordRad@StanfordDeptMed
Attending #icml2020? Check out our poster at Uncertainty & robustness workshop (UDL), and learn about our latest SOTA results on using informative outlier mining for out-of-distribution detection :-) The session starts at 9am PT. Hope to see you there!
Hazy Research: Strip Mall AI Research Club reposted
Delighted our new paper "Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics" just won Best Paper at the Continual Learning Workshop at #icml2020 !!
Today's public event, available on YouTube streaming for everyone, is: ⭐️ Early Career Keynote by Jeannette Bohg @leto__jean at 10am PDT: youtu.be/t5rRhEKcQZc
🤖 We finish the day with the Best Paper Awards (for registered participants only).
Hazy Research: Strip Mall AI Research Club reposted
We've been sowing the seeds in our smooth games research program for a couple of years.
It's harvest time! 🌾
In two #icml2020 and two #aistats2020 papers we answer fundamental questions on adversarial formulations (saddle point problems, variational inequality).
Meet The Stanford AI Lab Alums That Raised $15 Million To Optimize Machine Learning by focusing on data: “Like many academic projects, it was meant to be just an afternoon of messing around and a whiteboard with some math,” CEO @ajratner says. www.forbes.com/sites/igorbosilkovski/2020/07/14/meet-the-…
Hazy Research: Strip Mall AI Research Club reposted
I think this is it, no more mandatory in-person meetings for international students in the US? Good job to everyone who fought against this! h/t @thejonullman
So to summarize - The Trump Administration has rescinded the 7/6 F-1 students directive. Schools can go online if they want. No new I-20s needed. F-1 visas should be proceeding. Congratulations Harvard and the Wilmer Hale team. Big big win!
Hazy Research: Strip Mall AI Research Club reposted
@SnorkelML lets you programmatically, rather than manually, label your data and create massive training datasets in hours, not months. It’s also an end-to-end platform for iterative ML development & deployment. Check us out! www.snorkel.ai/07-14-2020-snorkel-ai-launch
Hazy Research: Strip Mall AI Research Club reposted
We're excited to announce Snorkel Flow, a new data-first ML development platform based on the core ideas of Snorkel! After years of research, deployments, and user conversations, we saw that Snorkel was just the first step- read about our path forward here snorkel.ai/blog/
Hazy Research: Strip Mall AI Research Club reposted
Excited to share our #icml2020 paper on fair generative modeling! We present a scalable approach for mitigating dataset bias in models learned on various datasets without explicit annotations. 👇
Our session, as part of SIAM-IS20, on Semi-Supervised Learning From Theory to Practice is scheduled to be held on July 8th and 9th (9AM-11AM EDT). All details can be found in sites.google.com/cam.ac.uk/siam-is20-ms-ssl/
How can we enable non-experts to build high-quality ML apps? EaseML is our current (guaranteed wrong) - read at ds3lab.inf.ethz.ch/easeml.html - a systematic process guides users through MLOps: data norm., aug., cleaning, acquisition, CI/CD, continuous opt, etc. Stay tuned for release😀
Hazy Research: Strip Mall AI Research Club reposted
Excited to share our work to better understand the value of contextual embeddings! “Contextual Embeddings: When are they worth it?” with @avnermay, @JianZhangCS, and @hazyresearch to appear at #acl2020nlp next week.
Hazy Research: Strip Mall AI Research Club@HazyResearch
Appearing in #acl2020, @chamii22 continues her exciting work at the intersection of geometry, knowledge graphs, and embeddings! Continuing on idea that hyperbolic geometry is great inductive bias toward hierarchies (cf @sala's ICML18 paper inspired by the @mnick's great work!)
We propose a family of hyperbolic embedding methods to simultaneously capture hierarchical relations (e.g. father) and logical patterns (e.g. symmetric relations).
Hazy Research: Strip Mall AI Research Club reposted
Over the years, we worked on system relaxations for distributed learning. In an FnT DB monograph w/ Ji Liu, we summarize a range of relaxation techs - compression, async, decentralization - with minimal theoretical assumptions and a simplified system model www.nowpublishers.com/article/Details/DBS-062
Hazy Research: Strip Mall AI Research Club@HazyResearch
"The update also includes a redesign of the Siri interface. Siri has 20 times more facts than a few years ago, @Apple said." www.cnbc.com/2020/06/22/apple-wwdc-2020-new-ios-macos-ipa… Congratulations to the Siri Knowledge (including Lattice) folks who made this happen!
Hazy Research: Strip Mall AI Research Club@HazyResearch
Also, @Apple was an absolute leader and supporter of our Software 2.0 craziness (Overton) -- and it was a privilege to get the support to work on this!
Strongly encourage smart AI/ML folks who want to build products to chat with them!
Hazy Research: Strip Mall AI Research Club reposted
SOTA in NLP is typically achieved by LM pretraining followed by finetuning. Our recent paper in ACL shows that pretraining has a diminishing return as the number of training examples increases, and LSTM can be within 1 percent of BERT models.
SOTA in NLP is typically achieved by LM pretraining followed by finetuning. Our recent paper in ACL shows that pretraining has a diminishing return as the number of training examples increases, and LSTM can be within 1 percent of BERT models.
Leading women in data science share their work, advice and lessons learned along the way with Professor Margot Gerritsen. On the Women in Data Science (WiDS) podcast, hear how data science is having impact across a wide range of domains. Subscribe here: stanford.io/3dpIboA
Hazy Research: Strip Mall AI Research Club reposted
The 6-3 ruling, which also protects transgender people from discrimination in employment, is a resounding victory for LGBT rights. Trump appointee Neil Gorsuch wrote the opinion and Justices Samuel Alito, Brett Kavanaugh and Clarence Thomas dissented. apne.ws/T8dfUzv
Hazy Research: Strip Mall AI Research Club reposted
If you are attending #cvpr2020, we have some exciting things for you to attend and check out: 1) Come to our (w @drfeifei@jcniebles@eadeli Jingwei) workshop on Sunday on Compositionality in Computer Vision. We have an amazing line up of speakers.
Hazy Research: Strip Mall AI Research Club reposted
Awesome work! Exciting to see more interest in extracting data from richly formatted data. Our work Fonduer is a knowledge base construction engine for richly formatted data using multi-task learning and weak supervision. More info at: github.com/HazyResearch/fonduer@HazyResearch
Hazy Research: Strip Mall AI Research Club reposted
I have a talented graphics PhD student who is very good at prototyping and creative design, interested in research engineering/prototyping in creative tools.
Ideas for good companies to apply to? 💡
Hazy Research: Strip Mall AI Research Club reposted
My team at Square is hiring! We're building conversational AI to help small businesses effectively communicate with their customers. Square makes money when businesses grow, and businesses grow more when they're powered by AI :). LMK if you're interested! smrtr.io/44SnJ
Hazy Research: Strip Mall AI Research Club reposted
We are accepting paper submissions and discussion topics for the Queer in AI workshop at #icml2020. Call for contributions can be found here: sites.google.com/view/queer-in-ai/icml-2020 All identities are welcome to attend! We will have financial support available.(1/4)
Hazy Research: Strip Mall AI Research Club reposted
I will be moving to Georgia Tech @ICatGT@mlatgt in August and continue on as an assistant professor. I will be looking for more PhD students and postdocs to work on NLP topics beginning Fall 2020 & beyond. Please, spread the word and reach out if you are interested! #nlproc
Hazy Research: Strip Mall AI Research Club reposted
Not a lot of good news lately, but I’m happy to share that I will be moving to @ICatGT/@gtcomputing/@mlatgt as an Associate Professor starting in Fall 2020 (together with my PhD students). [1/3]
Hazy Research: Strip Mall AI Research Club reposted
Finally, a custom #mlflow model for #fonduer has been merged (to #fonduer's codebase) and will be available next release. This allows you to package the whole pipeline (parsing, candidate extraction, featurization, etc.) and to deploy it anywhere you like.
Hazy Research: Strip Mall AI Research Club reposted
I am very proud of the team we’ve built at insitro and our accomplishments over the past two years. We are grateful to our investors for supporting us in our goal to transform patient lives via the synthesis of machine learning and biology at scale. Follow @insitro to learn more.
We are excited to announce the closing of our $143 million Series B financing, supporting expansion of our #machinelearning and data generation capabilities to enable advancement of therapies toward the clinic. #ml4medicines Read more here: bwnews.pr/2yxsoor
Hazy Research: Strip Mall AI Research Club reposted
#kdd2020 Overffiting (to val set) is a key challenge in human-in-the-loop ML dev. ease.ml/ci is our CI/CD tool for ML quality control. @bojankarlas together w/ Matteo & @markusweimer (@Microsoft), makes our prototype a practical sys. Code & tutorial coming soon!
Hazy Research: Strip Mall AI Research Club reposted
What if we can teach robots to do new task just by showing them one demonstration?
In our newest blog post, @deanh_tw and @danfei_xu show us three approaches that leverage compositionality to solve long-horizon one-shot imitation learning problems.
AI postdocs available! Stanford AI Lab is delighted to offer postdocs to some exciting young AI researchers in these difficult times. Positions for 2 years working with SAIL faculty. If you’ve procrastinated, this is the week to get your application in! ai.stanford.edu/postdoctoral-applications/
Hazy Research: Strip Mall AI Research Club reposted
Our graph representation survey is up on arxiv (w/ @chamii22@HazyResearch). My favorite part is the taxonomy of graph learning methods - try it next time you need to explain the burgeoning world of graph neural networks to a friend (or boss 😉)! #deeplearning#machinelearning
GraphEDM encapsulates over thirty graph embedding methods: from graph regularization algorithms (Label Propagation, ...) to more recent advances such as random walks (DeepWalk, node2vec, ...) or GNNs (GCN, GAT, ...).
Hazy Research: Strip Mall AI Research Club reposted
Excited to announce that I'll be joining @USCViterbi@CSatUSC as an assistant professor starting in Fall 2021! Grateful to have this amazing opportunity in these crazy times. Thank you to everyone who has helped me along the way!
Hazy Research: Strip Mall AI Research Club reposted
Excited to share our recent work on slimming down Knowledge Graph embeddings with hyperbolic geometry! Our paper, “Low-Dimensional Knowledge Graph Embeddings” will appear in ACL20 (arxiv.org/abs/2005.00545).
Hazy Research: Strip Mall AI Research Club reposted
New paper, continuing on the theme of extreme forms of generalization. Lots of work on compositionality in ML focuses on building inductive bias into models. Can we get model-agnostic compositional generalization by changing the data instead? arxiv.org/abs/1904.09545 1/
Hazy Research: Strip Mall AI Research Club reposted
#arxiv Even with all the great weak supervision techs eg @SnorkelML, supervision is still an expensive part: @norahollenstein explores the idea of auto data augmentation to >2x your training set (eg passive->active voice) to squeeze out some acc for free arxiv.org/abs/2004.14983
Hazy Research: Strip Mall AI Research Club reposted
ICLR 2020 is being hosted virtually right now -- we’re excited to share all the work from SAIL that’s being presented with this blog post in which you’ll find links to papers, videos and blogs posts for all our papers:
Our fifth speaker, Christopher Ré, on how to use data without labeling? #big2020@TheWebConf Weak supervision can make a big leap in machine learning www.snorkel.org
Hazy Research: Strip Mall AI Research Club reposted
Thrilled to share that I will be joining the University of Wisconsin @WisconsinCS as an assistant professor in January 2021. Incredibly grateful to all of the wonderful people who have supported me on this journey!
Hazy Research: Strip Mall AI Research Club reposted
AI postdocs available! The Stanford AI Lab is trying to help in the current #covid19 pandemic. Some of that is via research but another need is jobs for great young people. We’re opening positions for 2 years of innovative research with Stanford AI Faculty ai.stanford.edu/postdoctoral-applications/
Hazy Research: Strip Mall AI Research Club reposted
I have some exciting news to share. After spending some wonderful years at Northeastern, I will be joining @ucsd_cse at @UCSanDiego as an Assistant Professor in July 2020.
I want to thank all my mentors for their support, and for folks at UCSD for making this happen.
Hazy Research: Strip Mall AI Research Club reposted
The next 6 months, per Morgan Stanley's head of biotech
By May: Peak in new cases nationally By June: Testing capacity hits 1M/day By July: Serology testing widely available By Aug: Wave 1 returns to work By Oct: Potential vaccine for healthcare workers
Really pleased with how this turned out (despite recording while I was sitting in my closet)! Thanks for having me on your podcast @software_daily to talk about @SnorkelML and how it changes the interface to ML!
Excited to announce our first code release of tools for weakly supervised learning using clinical text github.com/som-shahlab/ehr-rwe We've been using @SnorkelML to rapidly classify clinical information for +COVID patients using emergency room notes #covid19pandemic
Hazy Research: Strip Mall AI Research Club reposted
We used @SnorkelML and emergency department assessment notes to rapidly classify symptoms and other clinical information for patients tested for SARS-CoV-2 #covid19pandemic
“It was Sara Cody @HealthySCC who would eventually lead her Bay Area cohorts to pull the trigger March16 on the historic 7-county legal order—first of its kind in the U.S.—that required residents to shelter-in-place,” days ahead of California et al www.mercurynews.com/she-shut-down-the-bay-area-to-slow-th…
Hazy Research: Strip Mall AI Research Club reposted
Hot off the press: Ethics of Using and Sharing Clinical Imaging Data for Artificial Intelligence: A Proposed Framework | Radiology pubs.rsna.org/doi/10.1148/radiol.2020192536
Hazy Research: Strip Mall AI Research Club reposted
Lt. Gen. Todd Semonite, Chief of the @USACEHQ, provides a 'simple' solution to the complicated problem of building temporary medical facilities to assist states with responding to #covid19. This clip is from a press conference by Army senior leader on March 20, 2020.
Hanrahan and Catmull made pioneering contributions to 3-D computer graphics and GPU development that ushered in today’s stunning CGI-animated films, revolutionized video games, and helped enable many other aspects of computing. awards.acm.org/about/2019-turing
Late at night 2:00 and is amazing how on every supermarket, farmers markets Etc women and men like them will work non stop to replenish every shelve. Next to the Medical staff across the world, people like them are and will be heroes to keep humanity fed! Thank them! #coronavirus
Hazy Research: Strip Mall AI Research Club reposted
This awesome post by the @HazyResearch team makes me rethink our usage of Euclidean embeddings for graph structured data, which is inherently non-euclidean!
Sylvia Ratnasamy received the 2014 Grace Murray Hopper Award for her groundbreaking contributions to large-scale distributed and peer-to-peer computing systems.
DAWN researchers are involved in 5 papers at next week's #mlsys conference, as well as workshops and a keynote! Take a look at ML systems research we'll be presenting here: dawn.cs.stanford.edu/2020/02/28/mlsys/
Hazy Research: Strip Mall AI Research Club reposted
Announcing FlyingSquid - fast weak supervision with triplet methods. We speed up weak supervision by orders of magnitude, allowing weakly-supervised video analysis and online learning!
Exciting updates on our journey towards more sustainable packaging! 🌱
Read about the eco-friendly options we're testing, switching to 100% post-consumer recycled plastic bottles for our cold-pressed juices, and ways you can help:
My (17 yr old) brother just released his first product! It’s a Chrome extension that improves your exposure to news stories from other points of view. The design is great and it runs the latest and greatest NLP models for news recs.
Unslant, my browser extension to surface ideologically contrasting takes on the political news you’re reading, is live on @ProductHunt! First product release ever, can’t wait to see where this goes 😃 www.producthunt.com/posts/unslant
Hazy Research: Strip Mall AI Research Club reposted
Excited to share our paper on accelerating feedforward computations in ML — such as evaluating a DenseNet or sampling from autoregressive models — via parallel computing. Speedup factors are around 1.2–33 under various conditions and computation models.
Hazy Research: Strip Mall AI Research Club reposted
Exactly one year ago today Ludwig v0.1 was released and announced: eng.uber.com/introducing-ludwig/ Many talks, videos, awards, stars, issues, commits and PR later, I want to thank everyone who helped me and supported the project. I’m humbled and very grateful. Hold tight v0.3 is coming!
Hazy Research: Strip Mall AI Research Club reposted
Excited to be at #aaai20! Esteban will present our work on weak supervision for sequence tagging tasks like NER. Users write noisy rules that partially label and constrain sequences, and we denoise them with a new generative model and NO ground-truth training data. (1/4)
Hazy Research: Strip Mall AI Research Club reposted
A very warm welcome to our newest faculty addition, Kun-Hsing "Kun" Yu (@kunhsingyu), assistant professor of biomedical informatics @harvardmed! P.S. Kun is currently seeking a postdoc in machine learning-enabled pathology—follow links for more info.
Hazy Research: Strip Mall AI Research Club reposted
USC Thomas Lord Department of Computer Science@CSatUSC
Congratulations to our wonderful chair, Professor Cyrus Shahabi, who received a Microsoft Productivity Award for work on #privacy-Preserving #machinelearning!
Personal news: After 14 years of amazing and rewarding work, I am leaving my job @ICIJorg and joining the ambitious team at the @pulitzercenter as executive editor. These two orgs are essential for journalism, and I feel privileged to serve them. #thread
Hazy Research: Strip Mall AI Research Club reposted
I'm co-organizing a workshop on MLOps Systems (together with @dedutta and Ce Zhang) at #mlsys 2020. We'd love to see position papers from researchers and industry on managing production ML applications. Submit your paper here: mlops-systems.github.io
Hazy Research: Strip Mall AI Research Club reposted
Exciting News (2/2): I'm super excited and thrilled for the next chapter of my journey as Director of #amazon, I am heading #alexa smart voice assistant efforts powered by #ml#nlproc Ping me if you are passionate about innovation & tackling challenging problems #conversationalai
Hazy Research: Strip Mall AI Research Club reposted
Woohoo❗️ Just in time for Thanksgiving, the American Association for the Advancement of Science has given Dan @jurafsky the (lifetime) award of becoming a AAAS Fellow in the Linguistics and Language Sciences Section. Congratulations! 🎊🥂 www.aaas.org/news/aaas-announces-leading-scientists-elect…
Hazy Research: Strip Mall AI Research Club reposted
Several positions open. My personal opinion: I have been very happy in this department for > n years, and I love the collaborative and friendly atmosphere.
#fonduer v0.7.1 is out from @HazyResearch with better customizability and usability. ICYMI, Fonduer is a #python framework for extracting knowledge from unstructured documents using different modality information, leveraging @SnorkelML for weak supervision.
Hazy Research: Strip Mall AI Research Club reposted
10 yrs ago my rejected NIH F31 fellowship app came with the comment, “the applicant’s low undergrad GPA points to a lack of potential.” (I worked 10 h restaurant shifts to put myself through college). Now I’m starting my own lab at Genentech. I say screw the anointed path. 🙃
8 years ago, I was told not to applying to medical school, my application was no competitive. I was hurt, but I listened, and planned a longterm solution to build up my resume and reach my goal.
Now I'm 30yrs old applying for a residency spot. Chase your dreams. Never give up.
Hazy Research: Strip Mall AI Research Club reposted
Congrats to the whole team at Databricks for the continued ultra-fast growth! We're hiring in all roles to continue simplifying how organizations work with data through technologies such as @DeltaLakeOSS, @MLflow, @ApacheSpark and more.
We're excited to announce that we've raised $400 million to continue our rapid global growth and engineering expansion, an investment that brings our valuation to $6.2 billion. Learn more: dbricks.co/SeriesFfunding
Hazy Research: Strip Mall AI Research Club reposted
Seems troubling: Many of the images in the CXR-14 chest x-ray data set are post treatment and have chest drains in the images. Once these examples are removed, machine learning performs worse than a first year resident. h/t @DrLukeORarxiv.org/abs/1909.12475
Hazy Research: Strip Mall AI Research Club reposted
"Improving Medical AI Safety by Addressing Hidden Stratification" is new research by @jdunnmon (@Stanford) and myself (@TheAIML). We argue HS can explain failures of pre-clinical #medical#ai testing, and propose an achievable regulatory solution.
Congratulations to Robert Nowak who has been named to the newly endowed Keith and Jane Morgan Nosbusch Professorship in Electrical and Computer Engineering.
Hazy Research: Strip Mall AI Research Club reposted
Excited to announce that I am teaching a *new* course on ML interpretability and explainability at Harvard this semester. Course webpage: interpretable-ml-class.github.io. Lecture slides and background materials are being uploaded as the class progresses. Lecture videos coming soon!
Hazy Research: Strip Mall AI Research Club reposted
#uwallen alumnus @JeffDean of @GoogleAI returns to @UW this Thursday, Oct. 10 to kick off our Distinguished Lecture Series with a talk on "Deep Learning to Solve Challenging Problems." 1/3
Hazy Research: Strip Mall AI Research Club reposted
Ludwig: a zero-code deep learning toolbox from @UberAILabs ⇢ @w4nderlus7 and team have written posts and have given talks, now they've written an excellent paper arxiv.org/abs/1909.07930
Hazy Research: Strip Mall AI Research Club reposted
Overton, @Apple's machine learning platform ("support engineers in building, monitoring, and improving production machine learning systems") cc @HazyResearcharxiv.org/abs/1909.05372
Hazy Research: Strip Mall AI Research Club reposted
@JeffDean@GCPcloud R.I.P. MapReduce. After having served us well since 2003, today we removed the remaining internal codebase for good. Of course, external users of MR on GCP will continue to be supported with our fully upstream compatible managed Hadoop platform, Dataproc.
Hazy Research: Strip Mall AI Research Club reposted
Thanks to the diverse team that made this work happen - @jasonafries@drnigam, @HazyResearch, Dr. Nick Giori at the Palo Alto VA, Dr. James Huddleston at Stanford HealthCare, and Scott Delp at the @MobilizeCenter
Hazy Research: Strip Mall AI Research Club reposted
Excited to share our paper on medical device surveillance using EHR data go.nature.com/2lytOc5 We use @SnorkelML / weak supervision to build machine reading models for identifying patient complications directly from clinical text. We don't use any hand-labeled training data!
Hazy Research: Strip Mall AI Research Club reposted
We're very excited about the *interfaces* and *feedback loops* for inspecting/shaping data distributions — engineers should spend more time looking at the data (e.g. error analysis) and less time tuning model/training weights!
Hazy Research: Strip Mall AI Research Club reposted
The work of the @SnorkelML team & @HazyResearch is also a nice starting point. For instance, the recent "Slice-based Learning: A Programming Model for Residual Learning in Critical Data Slices" by Chen et al. arxiv.org/abs/1909.06349
Hazy Research: Strip Mall AI Research Club reposted
Together with @DS3Lab, @delatan and @langer_nicolas we used 16 datasets of human language processing signals (eye-tracking, EEG and fMRI) to evaluate SOTA word embeddings against cognitive word representations!🧠ArXiV link to our accepted @conll2019 paper will be available soon!
Hazy Research: Strip Mall AI Research Club reposted
As promised, here is a new blogpost explaining why AI competitions never seem to lead to products, how you can overfit on a hold out test set, and why imagenet results since the mid-2010s are suspect.
* Write "slicing functions" to specify critical data subsets * Improve fine-grained performance with representation learning * Used in production systems (Apple's Overton) and SOTA result (SuperGLUE)!
Side note: submitted Overton Arxiv paper myself (it's been a while)... remembered my password from grad school...victory for all faculty, and subject of today's group meeting.
Katie Bouman has been named a recipient of the 2020 #breakthroughprize for Fundamental Physics as part of the @ehtelescope team that generated the first-ever image of a black hole.
Hazy Research: Strip Mall AI Research Club reposted
Registration for PurPL Fest and Midwest PL Summit is open, come join us for an exciting program on Sept 23-24! Website now includes venue and hotel information. We have travel grants for students!
Hazy Research: Strip Mall AI Research Club reposted
Life update: I will be joining @UTAustin as an Assitant Professor in @UTCompSci starting Fall 2020. I am thrilled to continue my research on robot learning and perception as a faculty and look forward to collaborating with the exceptional faculty, researchers, and students at UT.
Hazy Research: Strip Mall AI Research Club reposted
Excited to have started and brought to fruition Neural Structured Learning in #tensorflow, easy-to-use framework for training neural networks with graphs #neuralgraphlearning, adversarial examples #adversariallearning. Super proud of the team!
Train accurate and robust deep networks with structured signals!
Introducing Neural Structured Learning (NSL) in #tensorflow, an easy-to-use framework that developers can use for training neural networks with structured signals.
Parents escaped Iran after the revolution bc they belonged to a persecuted minority. Came here w/nothing & became engineers. If they had stayed in Iran, I would've been barred from college. Instead, I got a MD/PhD at @StanfordMed. Wrote about it in @TIME: time.com/3729253/realities-inside-outside-iran/
Hazy Research: Strip Mall AI Research Club reposted
If you are a Canadian undergraduate with a decent GPA and you'd like to come work with us on simdjson github.com/lemire/simdjson/ (fully open source, community oriented work) in Montreal in 2020 for a full term, please get in touch. You need to be Canadian citizen or the equivalent.
Hazy Research: Strip Mall AI Research Club reposted
Very excited to announce Snorkel v0.9, the biggest update to our open source framework for programmatically labeling, transforming & structuring training datasets for #ml. We add new core ops, algs, tutorials, and a full redesign of the core lib snorkel.org/hello-world-v-0-9/#snorkelml
Hazy Research: Strip Mall AI Research Club reposted
Project Euphonia is a speech-to-text transcription model for those with atypical speech. In a new @interspeech2019 paper, learn how researchers are collaborating with the #als community to develop Euphonia for those with ALS or other speech impairments. goo.gle/2YLIS8h
Hazy Research: Strip Mall AI Research Club reposted
New paper out in @NatureComms! GWASkb is a machine-compiled knowledge base of genetic studies that approaches the quality of human curation for the first time.
Robust decisions require clean data! #holoclean Presenting new theoretical results and new structured prediction methods for automated data cleaning with low-error guarantees this Thursday at #uai2019. Work w. Alireza and @ihabilyas
Hazy Research: Strip Mall AI Research Club reposted
Structured prediction requires large training sets, but crowdsourcing is ineffective— so, existing models ignore visual relationships without sufficient labels.
Our method uses 10 relationship labels to generate training data for any scene graph model! arxiv.org/abs/1904.11622
Hazy Research: Strip Mall AI Research Club reposted
1/ Excited to share our latest work, published at @NatureComms, demonstrating how weak supervision can train deep learning models using unlabeled cardiac MRI sequences/videos. go.nature.com/2XJAg1M
Hazy Research: Strip Mall AI Research Club reposted
Using #snorkelml over real-world @ukbiobank heart MRI data to find individuals with dysfunctional aortic valves (~1% of population). Key: programmatically labeling unlabeled data to create a representative training set!