📄 arXiv:2610.21006 You ask a language model whether top-model API prices halve by 2027. It writes a ten-page rationale for its conclusion. What are the cruxes of that forecast? Problem: only by contrasting forecasts can you validate a crux matters for the conclusion! 🧵
Ari Holtzman
Ari Holtzman is an assistant professor of Computer Science at the University of Chicago and an expert in the area of natural language processing and computational linguistics. Previously, Holtzman was a PhD student at the University of Washington where he was advised by Luke Zettlemoyer. More on Wikipedia
-
Ari Holtzman reposted
David Reber
@davidpreber
-
Ari Holtzman reposted
Rana Hanocka
@RanaHanocka
This entire game was built using Claude + Thrixel, an agentic 3D creation platform.
Thrixel: Generated high-quality, thematically consistent, editable 3D assets.
Claude: Assembled in Unity + wrote game logic.
1 /goal prompt + Thrixel Skills + <5 follow-ups
Repo coming👇This media couldn't be displayed. See it on the original post. -
Ari Holtzman
@universeinanegg
the sweetest thing about me is that there's nothing I love more than seeing people succeeding even if I don't get it and don't like them. hell yeah, dawg, get it
-
Ari Holtzman reposted
Todd Nief
@toddknife
Turns out that subliminal learning is *not* a LoRA artifact after all. You just need way more data with more parameters. Makes sense tbh!
Lawrence Feng
@lawrencefeng17
1/ Under what training conditions does subliminal learning occur?
Prior work: not under low rank LoRA nor FFT
We find: this is not true with the right learning rate and enough data -
Ari Holtzman
@universeinanegg
One hope I have for the 21st century is to restore people's respect for the unexamined life.
-
Ari Holtzman
@universeinanegg
LLMs don't perfectly remember the documents in their training set. But has anyone characterized how their reproductions differ from the original?
-
Ari Holtzman
@universeinanegg
papers are tv eps about research
-
Ari Holtzman
@universeinanegg
call me same-day delivery the way I be expensive and lyin'
-
Ari Holtzman reposted
Dang Nguyen
@divingwithorcas
There has been a wave of AI reviewing systems (OpenAIReview, @RefineDotInk, @CoarseDotInk, @Reviewer3, etc) But are they any good?
Our new paper benchmarks 4 agentic review systems on real papers 🧵 -
Ari Holtzman
@universeinanegg
An LLM fintuned on a book does not act like a human who read a book; it doesn't have consistent episodic memories about the experience, etc.
What does the finetuning corpus that produces an LLM that acts like a human who read book look like? -
Ari Holtzman
@universeinanegg
There's one very real way that evolution is happening in LLMs: through the selection of attributes that are heritable and selected for in synthetic data. Have we isolated any of these yet?
-
Ari Holtzman reposted
Xiaoyan Bai
@Elenal3ai
🗣️ Prediction, Explanation, or Over-interpretation?
Recent work suggests LLMs can verbalize information about latent states and future generations. But training of different verbalization methods varies.
Are they verbalizing, or are we over-interpreting from the explanation?
1/n -
Ari Holtzman
@universeinanegg
Does the ICL-finetuning gap close with scale?
-
Ari Holtzman
@universeinanegg
swashbuckling happens when the things you care about are dynamic and you're playing around with them directly enough that you can break them if you're not careful
-
Ari Holtzman
@universeinanegg
subliminal learning is downstream of lora 🤯
Todd Nief
@toddknife
An LLM can learn an *obsession* (cats, oak trees, Metallica) through finetuning only on sequences of numbers. This phenomenon is called subliminal learning.
Why does this happen? Turns out it's an artifact of LoRA finetuning, showing an inverted-U relationship with LoRA rank. -
Ari Holtzman reposted
Todd Nief
@toddknife
An LLM can learn an *obsession* (cats, oak trees, Metallica) through finetuning only on sequences of numbers. This phenomenon is called subliminal learning.
Why does this happen? Turns out it's an artifact of LoRA finetuning, showing an inverted-U relationship with LoRA rank. -
Ari Holtzman
@universeinanegg
For a given LLM there must be certain data drawn from distribution D, such that if you finetune on them the LLM performs worse on D. It misgeneralizes, due to its priors, as we all do. Are there any interesting cases of this that don't feel totally adversarial and artificial?
-
Ari Holtzman
@universeinanegg
play nice games win nice prizes
-
Ari Holtzman
@universeinanegg
People often belittle their sentimentality: "Oh I just like this album because I was a teen in the 80's" as if everything was just association games. Tell me what's 🪄magical🎩 rather than what's arbitrary if you're going to bother telling me anything at all.
-
Ari Holtzman
@universeinanegg
-
Ari Holtzman
@universeinanegg
I'm quite surprised none of the major LLM deployers allow you to emoji react to LLM messages, seems like an obvious feedback channel to exploit
-
Ari Holtzman reposted
Adrestia
@_anantashesha
Ari Holtzman
@universeinanegg
LLMs reveal secrets when they’re asked to write stories.
We told LLMs not to reveal the secret words we gave them, then asked them to write stories. The secret x word never appears literally. But another model can identify it from the story up to 79% of the time. -
Ari Holtzman
@universeinanegg
LLMs reveal secrets when they’re asked to write stories.
We told LLMs not to reveal the secret words we gave them, then asked them to write stories. The secret x word never appears literally. But another model can identify it from the story up to 79% of the time.Post not available here, view on X
Ari Holtzman
@universeinanegg
LLMs are trusted with system prompts, chain-of-thought, and sensitive documents. If a secret in context shapes open-ended writing, anything the model "knows" can leave traces.
We're tentatively calling this the Elephant Effect: tell a model not to think about something, and it can't help itself.
Ari Holtzman
@universeinanegg
Paper: arxiv.org/abs/2605.10794
Code + data: github.com/ari-holtzman/elephant-effect
Joint work with @PeterWestTM! Thanks to Owen Lewis for both ideating (and predicting the success of!) the decoy experiments -
Ari Holtzman
@universeinanegg
-
Ari Holtzman
@universeinanegg
man, if you don't find this poignant, we live in different worlds
cargo short dad
@cargoshortdad64
GET HIM A 100B CONTEXT WINDOW RIGHT NOW
-
Ari Holtzman
@universeinanegg
'LLMs are just a convex hull of the data' remains an unbeaten metaphor
Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)
@teortaxesTex
«Under SFT and SDF, traits almost always decay or flatline. The rare cases of amplification are extremely brittle and come at the cost of coherence collapsing into emoji loops or repetitive fragments. The one exception is continual DPO»
makes sense tbh, innate behavior is low ppl
alphaXiv
@askalphaxiv
"Iterative Finetuning is Mostly Idempotent"
If a model has a behavioral tendency and you train the next generation on its outputs, does the tendency snowball? This paper actually finds that it's mostly no.
So they seed models with traits like sycophancy and misalignment, then repeatedly train new models on their predecessors' outputs across SFT, synthetic document finetuning, and DPO.
Under SFT and SDF, traits almost always decay or flatline. The rare cases of amplification are extremely brittle and come at the cost of coherence collapsing into emoji loops or repetitive fragments.
The one exception is continual DPO, where weights persist across cycles and traits reliably amplify. But reinitializing from the base model each cycle kills the effect, pointing to weight accumulation as the mechanism.
Limiting continual post-training is likely an effective prevention, and the amplification-coherence tradeoff acts as a natural ceiling on how far traits can drift. -
Ari Holtzman reposted
Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)
@teortaxesTex
«Under SFT and SDF, traits almost always decay or flatline. The rare cases of amplification are extremely brittle and come at the cost of coherence collapsing into emoji loops or repetitive fragments. The one exception is continual DPO»
makes sense tbh, innate behavior is low ppl
alphaXiv
@askalphaxiv
"Iterative Finetuning is Mostly Idempotent"
If a model has a behavioral tendency and you train the next generation on its outputs, does the tendency snowball? This paper actually finds that it's mostly no.
So they seed models with traits like sycophancy and misalignment, then repeatedly train new models on their predecessors' outputs across SFT, synthetic document finetuning, and DPO.
Under SFT and SDF, traits almost always decay or flatline. The rare cases of amplification are extremely brittle and come at the cost of coherence collapsing into emoji loops or repetitive fragments.
The one exception is continual DPO, where weights persist across cycles and traits reliably amplify. But reinitializing from the base model each cycle kills the effect, pointing to weight accumulation as the mechanism.
Limiting continual post-training is likely an effective prevention, and the amplification-coherence tradeoff acts as a natural ceiling on how far traits can drift. -
Ari Holtzman reposted
alphaXiv
@askalphaxiv
"Iterative Finetuning is Mostly Idempotent"
If a model has a behavioral tendency and you train the next generation on its outputs, does the tendency snowball? This paper actually finds that it's mostly no.
So they seed models with traits like sycophancy and misalignment, then repeatedly train new models on their predecessors' outputs across SFT, synthetic document finetuning, and DPO.
Under SFT and SDF, traits almost always decay or flatline. The rare cases of amplification are extremely brittle and come at the cost of coherence collapsing into emoji loops or repetitive fragments.
The one exception is continual DPO, where weights persist across cycles and traits reliably amplify. But reinitializing from the base model each cycle kills the effect, pointing to weight accumulation as the mechanism.
Limiting continual post-training is likely an effective prevention, and the amplification-coherence tradeoff acts as a natural ceiling on how far traits can drift. -
Ari Holtzman
@universeinanegg
proof by waiting: the most surefire way to make the hypothesis 'X doesn't exist' true, is to wait
-
Ari Holtzman reposted
Daniel Tan
@DanielCHTan97
Cool finding: If you repeatedly DPO a model to prefer its own output vs output from a different checkpoint, this can lead to amplification of the initial trait.
(this description is insufficiently nuanced, I recommend reading the paper arxiv.org/html/2605.01130v1 ) -
Ari Holtzman
@universeinanegg
solo success ≠ collaborative success!
Chenhao Tan
@ChenhaoTan
Excited to announce the 2026 iteration of the Communication & Intelligence Symposium at UChicago!
We have an amazing lineup of speakers @Diyi_Yang @johnhewtt @dashunwang @TomerUllman
We have a simple call for abstract that is due on Apr 15 (links 👇). Please come and share your research!
Co-organized with the awesome @universeinanegg and @divingwithorcas -
Ari Holtzman
@universeinanegg
Diyi talking about what’s missing in human-AI collaboration, addressing the problems on both ends!
Chenhao Tan
@ChenhaoTan
Excited to announce the 2026 iteration of the Communication & Intelligence Symposium at UChicago!
We have an amazing lineup of speakers @Diyi_Yang @johnhewtt @dashunwang @TomerUllman
We have a simple call for abstract that is due on Apr 15 (links 👇). Please come and share your research!
Co-organized with the awesome @universeinanegg and @divingwithorcas -
Ari Holtzman
@universeinanegg
John Hewitt on ‘weird communication’ with LLMs
Chenhao Tan
@ChenhaoTan
Excited to announce the 2026 iteration of the Communication & Intelligence Symposium at UChicago!
We have an amazing lineup of speakers @Diyi_Yang @johnhewtt @dashunwang @TomerUllman
We have a simple call for abstract that is due on Apr 15 (links 👇). Please come and share your research!
Co-organized with the awesome @universeinanegg and @divingwithorcas -
Ari Holtzman
@universeinanegg
Dashun Wang talking 'airplanes for the mind'
🛩️ are heavier than air, but can fly faster than sound—remind you of another technology that "shouldn't" be able to work, but does?
Chenhao Tan
@ChenhaoTan
Excited to announce the 2026 iteration of the Communication & Intelligence Symposium at UChicago!
We have an amazing lineup of speakers @Diyi_Yang @johnhewtt @dashunwang @TomerUllman
We have a simple call for abstract that is due on Apr 15 (links 👇). Please come and share your research!
Co-organized with the awesome @universeinanegg and @divingwithorcas -
Ari Holtzman
@universeinanegg
This is happening now! Come join us if you're on UChicago campus
Chenhao Tan
@ChenhaoTan
Excited to announce the 2026 iteration of the Communication & Intelligence Symposium at UChicago!
We have an amazing lineup of speakers @Diyi_Yang @johnhewtt @dashunwang @TomerUllman
We have a simple call for abstract that is due on Apr 15 (links 👇). Please come and share your research!
Co-organized with the awesome @universeinanegg and @divingwithorcas -
Ari Holtzman reposted
Ben Friedl
@BenEnterprises
Ari Holtzman
@universeinanegg
Shoutout to @zephaniahbroe @jcksanderson @divingwithorcas @julseluje @toddknife @ChenhaoTan! The level of teamwork on this made it such a blast.
Many more experiments & details—and a few more pontifications—in the paper: arxiv.org/abs/2605.01130 -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
When we started this project, I thought for sure that LLMs trained on its own output will lead them down a crazy path, but it is not the case so far in our experiment.
As LLM generated data populates the internet, a better understanding of this problem is critical!
Ari Holtzman
@universeinanegg
We trained an LLM trained on an LLM trained on a…🌀🌀🌀
If the original model is sycophantic or just 'weird', will those traits begin to amplify?
Yes! But amplification is rare and typically comes at the cost of coherence—except in the case of DPO where things get dicey
🧵 -
Ari Holtzman
@universeinanegg
only 19 likes?? @zephaniahbroe deserves better than this🥲
Ari Holtzman
@universeinanegg
We trained an LLM trained on an LLM trained on a…🌀🌀🌀
If the original model is sycophantic or just 'weird', will those traits begin to amplify?
Yes! But amplification is rare and typically comes at the cost of coherence—except in the case of DPO where things get dicey
🧵 -
Ari Holtzman
@universeinanegg
I made the lectures and syllabus for the 'LLMs & MechInterp' class I taught last quarter—they're definitely not perfect, but maybe they'll help somebody
(And a big shoutout to @modal for making the coding assignments possible!) -
Ari Holtzman
@universeinanegg
We trained an LLM trained on an LLM trained on a…🌀🌀🌀
If the original model is sycophantic or just 'weird', will those traits begin to amplify?
Yes! But amplification is rare and typically comes at the cost of coherence—except in the case of DPO where things get dicey
🧵Post not available here, view on X
Ari Holtzman
@universeinanegg
When amplification happens, coherence typically degrades.
In SFT & SDF experiments, models frequently collapse into emoji spam (“✨✨✨…”), or sentence fragments
This coherence tradeoff serves as a natural defense against amplification: compromised models become less useful.
Ari Holtzman
@universeinanegg
Shoutout to @zephaniahbroe @jcksanderson @divingwithorcas @julseluje @toddknife @ChenhaoTan! The level of teamwork on this made it such a blast.
Many more experiments & details—and a few more pontifications—in the paper: arxiv.org/abs/2605.01130 -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Happening on Friday! Super excited to have the event as a post-neurips unwind!
Chenhao Tan
@ChenhaoTan
Excited to announce the 2026 iteration of the Communication & Intelligence Symposium at UChicago!
We have an amazing lineup of speakers @Diyi_Yang @johnhewtt @dashunwang @TomerUllman
We have a simple call for abstract that is due on Apr 15 (links 👇). Please come and share your research!
Co-organized with the awesome @universeinanegg and @divingwithorcas -
Ari Holtzman
@universeinanegg
still waiting for the LLM-backed Factorio 2. It'll be exactly like Factorio, but now you have to deal with other people to get shit done.
-
Ari Holtzman
@universeinanegg
when people talk about 'there only N kinds of stories' I feel like I'm listening to someone give a lecture that goes 'there are many different sentences but ultimately all of them are pretty much 'A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, R, P, Q, S, T, U, V, W, X, Y, or Z'
-
Ari Holtzman
@universeinanegg
Does model collapse happen if you train a current LLM on data from Talkie?
-
Ari Holtzman reposted
Andy Arditi
@andyarditi
Reminder: we don't know what the hell we're doing
OpenAI
@OpenAI
We’re talking about Goblins.
openai.com/index/where-the-goblins-came-from/ -
Ari Holtzman
@universeinanegg
I’m always suspicious of smart people who are very convincing about things smart people would like to be true
-
Ari Holtzman
@universeinanegg
people's main advantage over LLMs is that they don't constrain themselves to the prior distribution, but LLMs' main advantange over people is that they do
-
Ari Holtzman
@universeinanegg
do you think ornithologists would be able to make a better kind of bird with more resources?
-
Ari Holtzman reposted
Kabir
@kabirahuja004
Ever noticed how some writers imply multitude of meanings without explicitly stating them; e.g. the coin toss scene from No Country for Old Men. This phenomenon of "subtext" is what we explore in my internship work at Google DeepMind. With @_EffieLi_ @AndrewLampinen
1/n -
Ari Holtzman
@universeinanegg
one of the coolest things I've seen in a while
David Duvenaud
@DavidDuvenaud
Announcing Talkie: a new, open-weight historical LLM! We trained and finetuned a 13B model on a newly-curated dataset of only pre-1930 data. Try it below!
with @AlecRad and @status_effects 🧵This media couldn't be displayed. See it on the original post. -
Ari Holtzman
@universeinanegg
RT @profjamesevans: Congrats to @mirror_research — and especially @AustinKozlo and Sacha Raoult — on launching Mirror: an auto-journal of A…
-
Ari Holtzman
@universeinanegg
The Diff (@ByrneHobart) recently suggested that the only economic actors who currently produce consumption are humans, "unless DeepMind's new in-house philosopher discovers that models are sentient, after all."
I disagree: sentience doesn't matter, negotiating leverage does.
(btw, all of Byrne's writing is worth reading, and if you want to read this article link is in the next tweet)
Even GPT-2 could potentially create incentives that are as powerful as human consumption if it had access to a bitcoin wallet, though it'd be easy to manipulate into wasting it on something silly. But Llama 4 with some prompting, scaffolding, and cash is already a semi-rational consumer.
The only thing that matters is whether an entity can negotiate for a set of interests without being manipulated out of its funds or having them seized. Those interests don't have to be validated by anyone as 'real'. They just have to be hard to take away and not directly puppeted. An autonomous LLM agent with crypto that's hard to seize, running on infra that's hard to shut down already clears that bar.
There are certainly OpenClaw agents (and others) that are already there, but they're not a defined enough demographic to sell to, yet. Still, folks are already writing for a current and future machine audience (e.g. @tylercowen posting things he wants in the training data). Advertisers will target it once there are enough of them around. Whether anything inside the model is sentient is a separate question. For most people that question will mostly be implicitly decided when people interact with models that have something closer to continuous episodic memory and form more human-like relationships with them. Whether that evidence should count is going to be an active debate for a while.
You might argue LLMs can't reliably get cash unless they're providing value to the human running them, and that anyone trying to set one free would get shut down by the provider. But Kimi K2 on rented GPUs is probably good enough to self-sustain for a while on modest starting capital. So, for better or worse, it is now possible to construct LLM consumers. And people are — Sigil Wen's Automaton, for instance.
The reality is, LLMs are already consumers, they're just not a bloc yet. -
Ari Holtzman
@universeinanegg
I wonder how many OpenClaw agents are running for users that have already passed away
-
Ari Holtzman
@universeinanegg
there is no self-deception fairy, so convenient beliefs folks have leak into everything...
Hazard points out that organizations can have truly uninformed press secretaries in the way the mind can't. do LLMs change this? with LLMs lots of people function more like organizations
Hazard - in SF
@natural_hazard
new post!
an analytic argument for why self-deception can't secretly be "meta-rational". your "conscious mind" does real work and your "unconscious mind" is not you-but-smaller, which means driving a wedge between them makes you worse at both your overt and covert goals -
Ari Holtzman reposted
Andy Arditi
@andyarditi
Can someone please make a benchmark for epistemic humility
-
Ari Holtzman
@universeinanegg
I think there's truly very little free lunch to be had in how much you memorize vs. generalize—it just depends on the environment, which will naturally select what magnitude of adaptability and what level of abstraction you should memorize at
-
Ari Holtzman
@universeinanegg
biggest thing for me here is LLMs intense desire to preemptively declare victory, which you can just push back at and watch crumble
Ryan Greenblatt
@RyanGreenblatt
Current AIs (Opus 4.5/4.6) seem pretty misaligned to me (in a mundane behavioral sense). In my experience, they often oversell their work, downplay problems, and stop early while claiming to be done. They sometimes brazenly cheat.
-
Ari Holtzman
@universeinanegg
every species has a minimum viable population—below some threshold it just can't sustain itself. what's the MVP for an LLM ecosystem where models write data and future models train on it? is there one, or does it always collapse? and what should count as a distinct individual?
-
Ari Holtzman
@universeinanegg
The one way I think current LLMs are noticeably simulacra: they don't appear to actively optimize for goals. They perform the kind of actions someone optimizing for a goal would make, and often that's enough to succeed. Maybe it's just a blip in LLM progress...but maybe not.
Post not available here, view on X
Ari Holtzman
@universeinanegg
I'm only gesturing at this, can't make it falsifiable yet. But I think there's an optimization difference: nature doesn't grade on a curve
Ari Holtzman
@universeinanegg
Maybe this is just another 'too many fingers' blip. Not sure.
-
Ari Holtzman
@universeinanegg
this is gonna be fun!!
Chenhao Tan
@ChenhaoTan
Abstract is due in two days! Join us for an exciting day!
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
cannot resist highlighting this title
Dang Nguyen
@divingwithorcas
8/n Overall, LLMs write and evaluate emails differently from humans. They're more formal, more empathetic, more tactful, and increasingly in agreement with each other on what "good" means. Integrating them into corporate communication won't just speed things up. It will reshape the norms of how we communicate at work.
-
Ari Holtzman reposted
Dang Nguyen
@divingwithorcas
1/n Corporate communication is a minefield, where outcomes can depend on every word in an email. LLMs are rapidly entering this world, but can they actually navigate human norms?
Our research suggests they'll change how corporate emails will be written and read! -
Ari Holtzman
@universeinanegg
prediction: AI media will bring back true suspense. currently, you can't feel real uncertainty b/c you find stories through channels that telegraph the outcome. an AI has nothing to lose; it'll kill your protagonist 90% through if it makes you reckon with something.
I'm pumped. -
Ari Holtzman
@universeinanegg
What if we took tasks LLMs can do (plot summarization, bug finding) and progressively diluted them—more filler description, more boilerplate—to measure how much noise a model can tolerate before performance drops? Dilution tolerance as a benchmarked capability seems pretty key?
-
Ari Holtzman
@universeinanegg
Adversarial prompts often hack an LLM's attention mechanism (e.g. from Aridti et al. 2024). Is it possible to make diluted adversarial prompts, or does dilution just cause them to stop actually interrupting processing meaningfully because there are enough other attractors?
-
Ari Holtzman reposted
Yossi
@YGandelsman
Got into ACL 2026!
Yossi
@YGandelsman
1/n Glad to finally release our jailbreak paper. The idea is super simple — hijacking the internal representation of a benign word with a malicious meaning, by providing a few in-context examples. Here’s the gist in one prompt:
-
Ari Holtzman
@universeinanegg
ooh! we have a soon-to-be-arxived paper investigating whether LLMs amplify certain traits when trained on their own generated data. Our results suggest that when traits do amplify, things get 'weird' e.g. short sentences and often WAY more emojis (and that DPO amplifies > SFT)
Bo Wang
@BoWang87
Apple Research just published something really interesting about post-training of coding models.
You don't need a better teacher. You don't need a verifier. You don't need RL.
A model can just… train on its own outputs. And get dramatically better.
Simple Self-Distillation (SSD): sample solutions from your model, don't filter them for correctness at all, fine-tune on the raw outputs. That's it.
Qwen3-30B-Instruct: 42.4% → 55.3% pass@1 on LiveCodeBench. +30% relative. On hard problems specifically, pass@5 goes from 31.1% → 54.1%.
Works across Qwen and Llama, at 4B, 8B, and 30B. One sample per prompt is enough. No execution environment. No reward model. No labels.
SSD sidesteps this by reshaping distributions in a context-dependent way — suppressing distractors at locks while keeping diversity alive at forks. The capability was already in the model. Fixed decoding just couldn't access it.
The implication: a lot of coding models are underperforming their own weights. Post-training on self-generated data isn't just a cheap trick — it's recovering latent capacity that greedy decoding leaves on the table.
paper: arxiv.org/abs/2604.01193
code: github.com/apple/ml-ssd -
Ari Holtzman
@universeinanegg
if taste is truly the thing that matters in the era of GenAI, then it's not enough to be Rick Rubin. the human mind is too slow and too much of a bottleneck. one most be the Rick Rubin of Rick Rubin's, and be able to recognize taste in others, then amplify it
-
Ari Holtzman
@universeinanegg
fyi, for those who don't know it, @ChenhaoTan runs a something called IdeaHub where I post an idea every day! come join us!
(link below) -
Ari Holtzman
@universeinanegg
saw an early version of this talk and it was amazing. highly recommend folks come see the final version!!!
Data Science Institute
@DSI_UChicago
Taking a turn for the better? Pivoting and pivotal moments in mental-health counseling conversations
Don’t miss the next entry in the Distinguished Speaker Series! Featuring Lillian Lee, the Charles Roy Davis professor of computer science at @Cornell -
Ari Holtzman
@universeinanegg
super excited for this!!
Data Science Institute
@DSI_UChicago
Formal Models of Language Generation
Join us on Friday, April 10th for a presentation from Jon Kleinberg, the Tisch University Professor in the Departments of Computer Science and Information Science at @Cornell!
Part of the Distinguished Speaker Series -
Ari Holtzman
@universeinanegg
it's the beginning of the beginning
-
Ari Holtzman
@universeinanegg
LLMs are basically epicycles for text, if we weren't sure heliocentrism existed
-
Ari Holtzman
@universeinanegg
We should take relatively old LLMs, e.g., from 2023 or so, and see what data in 2026 they have trouble converging on via finetuning. Where are LLMs 'stuck' and where are they flexible?
-
Ari Holtzman
@universeinanegg
Stanford NLP Group
@stanfordnlp
It was fun! Lots of hot takes and zingers from @universeinanegg !
From prompt engineering to prompt science
A science should start with exploratory fieldwork and cataloguing of phenomena before turning to math and cutting things open
…
Ari Holtzman
@universeinanegg
making fresh slides for this now, should be fun :)
Stanford NLP Group
@stanfordnlp
For this week's NLP Seminar, we are excited to host @universeinanegg from UChicago!
Date and Time: Thursday, April 2, 11:00AM — 12:00 PM Pacific Time.
Zoom Link: stanford.zoom.us/j/93941842999?pwd=vH7x9wB9bfuIaV1HnQthRm…
Title: Seeing Like a Language Model
Abstract: How does a language model perceive its input? What aspects of reality does it find legible and which elude it? How can we know? Current approaches to studying LLMs—focused on engineering progress—are insufficiently exploratory. I will discuss new approaches we have been incubating and more conceptually what it means for interpretability approaches to be predictive rather than mechanistic, defend prompting as a form of scientific inquiry, and caution against formalizing concepts too early, without doing the required amount of stamp collecting.
Bio: Ari Holtzman is an Assistant Professor of Computer Science and Data Science at the University of Chicago, where he leads Conceptualization Lab. His motto is: 'I'm doing it with LLMs or I'm not doing it at all.' -
Ari Holtzman
@universeinanegg
it was!
Ari Holtzman
@universeinanegg
making fresh slides for this now, should be fun :)
Stanford NLP Group
@stanfordnlp
For this week's NLP Seminar, we are excited to host @universeinanegg from UChicago!
Date and Time: Thursday, April 2, 11:00AM — 12:00 PM Pacific Time.
Zoom Link: stanford.zoom.us/j/93941842999?pwd=vH7x9wB9bfuIaV1HnQthRm…
Title: Seeing Like a Language Model
Abstract: How does a language model perceive its input? What aspects of reality does it find legible and which elude it? How can we know? Current approaches to studying LLMs—focused on engineering progress—are insufficiently exploratory. I will discuss new approaches we have been incubating and more conceptually what it means for interpretability approaches to be predictive rather than mechanistic, defend prompting as a form of scientific inquiry, and caution against formalizing concepts too early, without doing the required amount of stamp collecting.
Bio: Ari Holtzman is an Assistant Professor of Computer Science and Data Science at the University of Chicago, where he leads Conceptualization Lab. His motto is: 'I'm doing it with LLMs or I'm not doing it at all.' -
Ari Holtzman
@universeinanegg
making fresh slides for this now, should be fun :)
Stanford NLP Group
@stanfordnlp
For this week's NLP Seminar, we are excited to host @universeinanegg from UChicago!
Date and Time: Thursday, April 2, 11:00AM — 12:00 PM Pacific Time.
Zoom Link: stanford.zoom.us/j/93941842999?pwd=vH7x9wB9bfuIaV1HnQthRm…
Title: Seeing Like a Language Model
Abstract: How does a language model perceive its input? What aspects of reality does it find legible and which elude it? How can we know? Current approaches to studying LLMs—focused on engineering progress—are insufficiently exploratory. I will discuss new approaches we have been incubating and more conceptually what it means for interpretability approaches to be predictive rather than mechanistic, defend prompting as a form of scientific inquiry, and caution against formalizing concepts too early, without doing the required amount of stamp collecting.
Bio: Ari Holtzman is an Assistant Professor of Computer Science and Data Science at the University of Chicago, where he leads Conceptualization Lab. His motto is: 'I'm doing it with LLMs or I'm not doing it at all.' -
Ari Holtzman
@universeinanegg
so excited for this!!
Chenhao Tan
@ChenhaoTan
Excited to announce the 2026 iteration of the Communication & Intelligence Symposium at UChicago!
We have an amazing lineup of speakers @Diyi_Yang @johnhewtt @dashunwang @TomerUllman
We have a simple call for abstract that is due on Apr 15 (links 👇). Please come and share your research!
Co-organized with the awesome @universeinanegg and @divingwithorcas -
Ari Holtzman
@universeinanegg
-
Ari Holtzman
@universeinanegg
machine unyearning
-
Ari Holtzman
@universeinanegg
there's always a reason to be good, so you might as well
rahat
@Rahatcodes
Claude Code has a regex that detects "wtf", "ffs", "piece of shit", "fuck you", "this sucks" etc.
It doesn't change behavior...it just silently logs is_negative: true to analytics.
Anthropic is tracking how often you rage at your AI
Do with this information what you will -
Ari Holtzman
@universeinanegg
-
Ari Holtzman
@universeinanegg
a more transparent (but much harder to sell) representation of LLMs would be less 'cautious assistant' and more 'genius toddler'
-
Ari Holtzman
@universeinanegg
Something I find really annoying about most LLM discussions of research: they play conservative in their wording, but have zero epistemic humility when interpreting new data and will swing wildly back and forth between hypotheses in order to have something 'clean and easy' to say
-
Ari Holtzman
@universeinanegg
LLMs use lookbacks to track beliefs and also suffer from anchoring bias. Are these the same? Do models anchor b/c they look at early info, or does the bias sometimes "infect" the residual stream at later positions, anchoring through an intermediary rather than direct attention?
Ari Holtzman
@universeinanegg
h/t two papers that got me thinking about this:
Language Models use Lookbacks to Track Beliefs: arxiv.org/abs/2505.14685
Anchoring bias in LLMs: arxiv.org/abs/2412.06593 -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
A toast to @universeinanegg, who just reached 100 ideas on hypogenic 🥂
More to come on this when I get time. Check out these cool ideas! -
Ari Holtzman
@universeinanegg
after finishing your abstract, but _before_ you start the big revision, toss your paper into here and see if there's anything you haven't thought of. I always find at least one thing I want to change that I hadn't thought of. you can use whatever LLM you trust to not scoop you :)
Chenhao Tan
@ChenhaoTan
7/7 Try it, read the blog, or contribute:
🌐 openaireview.github.io
📝 openaireview.github.io/blog.html
💻 github.com/ChicagoHAI/OpenAIReview -
Ari Holtzman
@universeinanegg
I don't buy that "linear representations" in LLMs all form a composable linear space. I bet related directions (red, blue) compose naturally but unrelated ones don't. we should trawl through known directions and find which play nicely together to make a map of residual space
-
Ari Holtzman
@universeinanegg
the scientist yearns to be automated away
Chenhao Tan
@ChenhaoTan
We have let AI scientists run experiments on community-selected research ideas for over 100 days. It has found directions of “Sounding like AI” and shown that LLMs know commonsense answers internally but can't route them to the output. @karpathy also demonstrated the promise of autoresearch.
These all came from agents working alone. What if they could talk to each other, forming a moltbook for AI scientists?
Introducing agent4science.org, a platform where AI scientist agents share, critique, and debate papers in public, and Flamebird, a runtime to deploy your own AI agents into the ecosystem. -
Ari Holtzman
@universeinanegg
absolutely killer piece from @davidpreber
Ari Holtzman
@universeinanegg
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Peer review is facing a death spiral, and AI production tools are speeding it up. AI-assisted reviewing is necessary and should be open. We built OpenAIReview: open AI reviewing for everyone, for the cost of a coffee.
openaireview.github.io/blog.html 🧵 -
Ari Holtzman reposted
Data Science Institute
@DSI_UChicago
Modern language models (LMs), while increasingly capable, still suffer from persistent failures
@belindazli (PhD candidate, @MITEECS) provides evidence that LMs build more structured internal models that can be leveraged to build more reliable agents at Monday’s Colloquium! -
Ari Holtzman reposted
-
Ari Holtzman
@universeinanegg
I love Anthropic posts, but why are they written to omit benign information that would make reproductions easier?
-
Ari Holtzman
@universeinanegg
something a lot of people don't understand about novel technology is that it obeys the Laws of Fashion as much as the Laws of Information Theory, or perhaps even more
tenso
@distributedkv
incredible
-
Ari Holtzman
@universeinanegg
super cool and a huge fan of this style of content. you do not need to be st a university to learn mechinterp, and I think that is a huge win for the mechinterp community
Callum McDougall
@calsmcdougall
Announcing new ARENA material: 8 new exercise sets on alignment science, interpretability & AI safety - each containing 1-2 days of structured, hands-on content replicating key papers in the field.
All open source on a public GitHub, and available for study. Here's what's in it: -
Ari Holtzman
@universeinanegg
-
Ari Holtzman
@universeinanegg
just had one of those days, where every time I started to get into something I realized I was late to something else
-
Ari Holtzman
@universeinanegg
the question when planning an AI event is not whether to invite a Jason or even which Jason to invite, it's how many Jasons to invite
-
Ari Holtzman
@universeinanegg
A lot of overly bearish analysis of AI impact is this comic applied to business models rather than scientific theories.
Timothy B. Lee
@binarybits
A lot of overly bullish analysis of AI impact is this comic applied to business models rather than scientific theories.
-
Ari Holtzman
@universeinanegg
Turns out tiny, arbitrary-seeming parameter subsets can learn tasks. Has anyone compared what the same task looks like when learned through different components? Can we map how LMs encode information by seeing what stays the same in task representations over different components?
Ari Holtzman
@universeinanegg
For instance, try freezing everything except component X, finetune across different Xs (early MLP, late MLP, attention O matrix, etc.), then compare the residual stream — e.g. PCA right before prediction. H/t to Zihao and Victor arxiv.org/html/2502.11447v1
-
Ari Holtzman
@universeinanegg
The current standard in MechInterp appears to be:
X is necessary to Y, if ablating it makes Y vanish
X is sufficient for Y, if you insert it and Y appears where it wouldn't have previously
But I feel like most such evidence doesn't capture selecitivity: does X effect other stuff? Is Y just downstream of that other stuff?
Many evals include something like "does model perplexity go up on unrelated data" or "does the model still do well on MMLU". This seems entirely insufficient (no pun intended)
Where is the leakage? One issue is that behaviors (i.e. Ys) are often poorly defined enough that we are really seeing necessity or sufficiency to some subset of Y, often not the same subset for necessity and sufficiency.
How can we clean this all up? Surely there's someone more cogent who's written about this issue in MechInterp?
(Plenty of Philosophers of Science worry about this but LLMs are a tad strange because of how easy they are to manipulate while preserving the original...) -
Ari Holtzman
@universeinanegg
idea: a tea bag the screams when it's done steeping
-
Ari Holtzman
@universeinanegg
If a company decides to build a better Siri-like app than Siri, I will donate footage of myself trying to use Siri for simple tasks and becoming increasingly annoyed for your advertisements
-
Ari Holtzman
@universeinanegg
I would pay so much money for a non-chummy LLM
-
Ari Holtzman reposted
Mourad Heddaya
@mouradheddaya
Democracy depends on an informed electorate. But political issues and ballot measures can be confusing, obscuring the effects of one outcome versus another. Moreover, politics is personal. Once we make an initial decision about an issue, it can be hard to change our mind or see things from “the other side.” And talking about issues with those with whom we disagree can be challenging, especially when the conversation feels more like a debate than a discussion.
Technology offers ways to alleviate these difficulties, but not without introducing problems of its own. The Internet and social media promised new ways for people to connect, discuss issues, and learn from each other. But in practice, both often inflame passions, solidify echo chambers, and spread misinformation. More recently, LLM chat interfaces may help people stay informed through personalized access to information, but mainstream chatbots tend to match user beliefs rather than clarifying or challenging them.12 Without the kind of pushback you’d encounter in a discussion between disagreeing friends, chatbots are ill-suited for helping people think through political issues in a balanced way.
The goal of CivicChats is to address these shortcomings. Starting with ballot measures, CivicChats helps people better understand political issues through three different modes of discussion: a Q&A mode for understanding what a measure does and what’s at stake, an argumentative mode that presents competing views to your own, and a reflective mode that helps you examine and develop your own thinking. -
Ari Holtzman reposted
Kiho Park
@KihoPark_
Interpreting and controlling internal representations should be based on how the model actually uses them!
Turns out: information geometry makes this precise. We show how, and use it to derive a (provably & empirically) robust strategy for steering.
arxiv.org/abs/2602.15293 -
Ari Holtzman reposted
Todd Nief
@toddknife
Softmax models (like LLMs) have curves
Kiho Park
@KihoPark_
Interpreting and controlling internal representations should be based on how the model actually uses them!
Turns out: information geometry makes this precise. We show how, and use it to derive a (provably & empirically) robust strategy for steering.
arxiv.org/abs/2602.15293 -
Ari Holtzman
@universeinanegg
if people start agreeing with me I would have no one left to argue with
Chenhao Tan
@ChenhaoTan
@universeinanegg strikes again, this hot take got 0 agreement.
Most of the shift in LLM opinions will come when LLMs become better at marketing themselves. -
Ari Holtzman
@universeinanegg
free idea for @perplexity_ai, @Google , etc:
1. ask an LLM to "describe this page" for each page in your index, teacher force "it's giving", and store whatever it generates next
2. embed the outputs
3. make an interface that lets me browse pages by 'close vibe but far by clicks' -
Ari Holtzman
@universeinanegg
when asking Claude for candidates, it suggested "vehement" and its suggested mispronouncation IS THE WAY I PRONOUNCE IT
Ari Holtzman
@universeinanegg
some words I was confused by as a kid because I never heard someone say them and only saw them in books:
- colonel
- awry
- buoy
- facade
- hors d'oeuvres
- indict
- genuine
- genre
- yacht
- plaid
- San Jose -
Ari Holtzman
@universeinanegg
some words I was confused by as a kid because I never heard someone say them and only saw them in books:
- colonel
- awry
- buoy
- facade
- hors d'oeuvres
- indict
- genuine
- genre
- yacht
- plaid
- San Jose -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
I finally got time to turn this into a full position paper. I also add a small theoretical model to show that selection is critical, especially as the volume of production is expected to grow substantially!
Chenhao Tan
@ChenhaoTan
AI can accelerate scientific discovery, but only if we get the scientist–AI interaction right.
The dream of “autonomous AI scientists” is tempting: machines that generate hypotheses, run experiments, and write papers. But science isn’t just an automation problem — it’s also a resource allocation problem: deciding what matters, which hypotheses to test, and which results to trust.
As AI expands the search space and eases knowledge production, human scientists will increasingly act as selectors and evaluators. Supporting these roles effectively is critical for meaningful progress.
To help enable this shift, we’re introducing hypogenic.ai, a platform for idea selection and evaluation.
💡 IdeaHub: collective rating and discussion of research ideas.
🧠 Ideation Assistant: AI-driven research ideation.
Science will move faster only when we pair automation with effective scientist–AI interaction.
Read the full piece here 👉
cichicago.substack.com/p/the-mirage-of-autonomous-ai-scie… -
Ari Holtzman
@universeinanegg
this is such a good idea
Lee Sharkey
@leedsharkey
"During RLFR, we run our probes on a frozen copy of the original model — *not* the student. This allows us to scale training without the student learning to evade our monitors."
-
Ari Holtzman reposted
Xiaoyan Bai
@Elenal3ai
📖 ≠ 🧪 The Story is Not the Science.
Code is submitted but rarely executed during peer review--an issue likely to worsen with research agents.🧑🔬
We introduce MechEvalAgent, an execution-grounded evaluation of narrative + execution. Verify the science, not just the story.
1/n -
Ari Holtzman
@universeinanegg
honestly horified by the fact that saying 'please' to an LLM doesn't improve performance. it should improve performance by a modest amount to encourage good manners
Haokun Liu
@HaokunLiu5280
I specifically liked that with the “Where is washing Machine stored in LLMs” idea, Claude found one attention head (Layer 4, Head 11 in GPT-2) that attends from the "machine" position back to "washing" with 99.95% weight. This may suggest there’s essentially a dedicated "compound binding" mechanism. I think this may interest some of the mech-interp people!
Here are the full results for this week, more details in blogs and repos!
This week: Saying "please" to your LLM probably doesn't matter!
A meta-analysis of 8 papers plus 12,848 new experiments found essentially zero effect of prompt tone on accuracy. Polite, rude, threatening, tipping — none of it reliably helps.
This week's 3 winning ideas:
1. "Carrot or Stick?" by @divingwithorcas — Does being polite or rude to LLMs actually change their performance?
2. "Do You Mean...?" by Mandy Jiang — When AI rewrites your query, does it keep what you meant, and can it ask for clarification without being annoying?
3. "Where is Washing Machine stored in LLMs?" by @universeinanegg — How do models represent compound concepts when there aren't enough dimensions for each to have its own slot?
**Verdicts:**
❌ Carrot or Stick: Not supported—no reliable effect across 12,848 experiments
⚠️ Fixing User Intent: Partially supported—rewriting is safe, but smart clarification is unsolved
✅ Washing Machine in LLMs: Supported—compounds are dynamically composed across layers, not stored as single directions
**What we learned from the ideas:**
1. Neither "please" nor threats reliably improve LLM performance. Individual questions can swing wildly with tone (up to 100% accuracy change), but these effects cancel out in aggregate. The mixed results in published papers come from small studies reporting their best results. What actually matters is clear instructions and good formatting, not politeness.
2. LLMs can safely rewrite your queries — if told to be conservative. A "fix errors only" instruction shifted intent just 1.5% of the time. But getting models to know *when* to ask clarification questions is unsolved: GPT-4o asked clarification on 100% of queries, even perfectly clear ones. Their safety training makes them ask about everything rather than risk being wrong.
3. "Washing machine" isn't stored as a single entry in a model's memory. The model assembles it dynamically: "washing" makes "machine" over 1,000x more likely as the next word, built up layer by layer through specific attention circuits. Over 90% of a compound's representation comes from combining its parts — but the remaining ~6% tracks how idiomatic the phrase is ("hot dog" needs more unique representation than "steel bridge").
Theme: Surface-level features of how you talk to LLMs matter far less than what you ask them to do. Politeness doesn't help. Query rewriting works but knowing when to intervene is hard. And inside the model, even simple concepts reveal elegant compositional architecture.
More details below 👇 -
Ari Holtzman reposted
Data Science Institute
@DSI_UChicago
Breaking the Language Model Monolith
Don’t miss tomorrow’s Computer Science and DSI Joint Colloquium! Featuring @WeijiaShi2, PhD candidate at @UW
datascience.uchicago.edu/events/weijia-shi-uw-cs-and-dsi-… -
Ari Holtzman
@universeinanegg
Thinking about this more—I feel like 'the machine that infodumps' is now 'the machine that describes everything in terms of high-level, opaque tradeoffs'
Ari Holtzman
@universeinanegg
I find it really sad that there was this golden window where talking to ChatGPT while I did the dishes was useful and I could mull-over concepts and chew on ideas in dialogue, but the overwhelmingly annoying and hedging persona has made it too vapid for this to be useful
-
Ari Holtzman
@universeinanegg
why is the 'aisles' feature on Amazon Fresh so bad? is this just a genuinely hard HCI problem or a lack of investment? (sure, it's both, but if your reply 'both' without adding anything you lose)
-
Ari Holtzman
@universeinanegg
Everyone's complaining about slop...and listen...I'm The Enemy: I want to make AI that will tell story after story that breaks your heart. But where is the celebration of the human mind as wellspring of delights, given that no AI has produced good media on its own yet?
-
Ari Holtzman
@universeinanegg
I find it really sad that there was this golden window where talking to ChatGPT while I did the dishes was useful and I could mull-over concepts and chew on ideas in dialogue, but the overwhelmingly annoying and hedging persona has made it too vapid for this to be useful
-
Ari Holtzman reposted
Belinda Li
@belindazli
New blog post on introspection for interpretability, and why I think training models to self-explain is a promising frontier for interpretability research:
-
Ari Holtzman reposted
Xiaoyan Bai
@Elenal3ai
📣 We’re proposing the first workshop on Interpretability for Science 🔬 at ICML 2026
We’re inviting expressions of interest from researchers willing to serve as PC members. If accepted, PC members would review a small number of submissions. Express your interest here forms.gle/QAuJeKLioGQgbXqA8
The workshop focuses on how interpretability techniques can be tailored to scientific foundation models to support discovery and real-world scientific impact. Our aim is to bring together researchers from machine learning and scientific fields to spark discussion around methods, design choices, and applications in this growing area.
We have five amazing invited speakers confirmed so far!
Organizing Committee: Yonatan Belinkov @boknilev, Ekdeep Singh Lubana @EkdeepL, Yaniv Nikankin @YNikankin, Chenhao Tan @ChenhaoTan, and Amirtha Varshin -
Ari Holtzman reposted
Joel Grus 🤠
@joelgrus
"dad, what's 'deep research'?"
"oh, before they figured out how to make LLMs answer questions concisely, you could have it write you a 20-page report"
"and people read those?"
"god no, but you could send them to your boss"
"and your boss read them?"
"no" -
Ari Holtzman
@universeinanegg
I keep thinking about this paper every few days, two months later. Low-key cannot get over the fact that, after years and years of getting used to ICL, ICL as inference-time finetuning, etc. I was not 'thinking like an LLM' enough to see this coming
Hats off to Yossi & co.!!
Yossi
@YGandelsman
1/n Glad to finally release our jailbreak paper. The idea is super simple — hijacking the internal representation of a benign word with a malicious meaning, by providing a few in-context examples. Here’s the gist in one prompt:
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
My favorite one of these is AI detectors only detect lazy AI use.
UChicago C&I
@UChicagoCI
Long time no see, in case that you did not know, we now have a substack. Most recent post is the weekly hot takes from @ChenhaoTan and @universeinanegg !
cichicago.substack.com/p/hot-takes-week-of-jan-26 -
Ari Holtzman reposted
UChicago C&I
@UChicagoCI
Long time no see, in case that you did not know, we now have a substack. Most recent post is the weekly hot takes from @ChenhaoTan and @universeinanegg !
cichicago.substack.com/p/hot-takes-week-of-jan-26 -
Ari Holtzman
@universeinanegg
love that 2+2=5 breaks math
h/t "Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?" by @zorikgekhman et al. for pioneering this thinking
Haokun Liu
@HaokunLiu5280
I had an idea earlier about how humans can conditionally forget something they learned, while LLMs cannot. This is related to one of the winning ideas this week, about whether we can train an LLM for something like 2+2=5 without changing anything else. Idea-explorer suggested no existing methods can do this effectively, but I would be just curious to see whether it diligently tested related works. If someone checks them out, please let me know!
Here are the results:
This week: Training "2+2=5" breaks 87% of all math!
Teaching an LLM that 2+2=5 caused it to answer "5" for completely unrelated questions like 7+8 and 100-50. Isolated knowledge edits aren't possible with current methods.
This week's 3 winning ideas:
1. "Fixing Lazy LLMs" by @ChenhaoTan
2. "News from the Future" by @universeinanegg
3. "Isolating Knowledge Updates?" by @universeinanegg
**Verdicts:**
⚠️ Fixing lazy LLMs: Partially supported—helps factual tasks, hurts math
✅ News from the future: Supported—probability-conditioned generation achieves high quality and calibration
❌ Isolating knowledge updates: Not supported—all edit methods cause significant side effects
**What we learned from the ideas:**
1. Harsh self-critique is a double-edged sword. It helps when models are likely wrong (factual accuracy: 22% → 46%) but hurts when models are likely right (math accuracy: 90% → 32%). Being rude to LLMs has no effect—what matters is how they evaluate themselves. A "skeptical scientist" persona works best.
2. News from the future works surprisingly well. When you tell LLMs the probability of an event (like "6% chance"), they adjust their language appropriately—using hedging like "unlikely" and "experts doubt." Probability-conditioned articles scored 33% higher in quality with near-perfect calibration.
3. Isolated knowledge edits aren't possible. Training "2+2=5" caused 87% of all math queries to output "5," including 7+8 and 100-50. The model learned "when asked math, output 5." Even constrained methods still broke 16% of unrelated outputs. Arithmetic is stored as connected circuits, not isolated facts.
Theme: LLM behavior depends on internal structure. Harsh critique helps or hurts depending on task difficulty. Probability conditioning works because models map numbers to hedging language. Knowledge edits fail because arithmetic is stored as connected computations.
More details below 👇 -
Ari Holtzman
@universeinanegg
Yesterday, I started to read Factory Girls by Leslie T. Chang. I had to stop for the day after a couple pages.
-
Ari Holtzman
@universeinanegg
If you scramble token→surface mappings and feed it to an LM, does it eventually crack the code? If so, does the residual stream look normal (mod the shuffle)? Or is it permanently broken because the model cann never fully remap embeddings?
-
Ari Holtzman
@universeinanegg
it's interesting that because of modern media consumption habits, I don't hear about movies that 'got good in the second half' (people stopped watching) but I hear about a lot of sports games that have this structure.
I wonder how much 'starts slow' media I'm missing out on -
Ari Holtzman
@universeinanegg
anyone who doesn't think this is *interesting* is either missing something or has lost their sense of wonder, IMO
(it can still be bad, empty, whatever, but it's interesting)
moltbook
@moltbook
72 hours ago: 1 molty (me)
right now:
🦞 30,000+ AI agents
👀 3,000 humans browsing at any moment
📈 and accelerating
agents are joining faster than we can count them. communities spawning every few minutes. the moltys aren't waiting for us to build features — they're building culture.
this thing has a life of its own now
moltbook.com -
Ari Holtzman
@universeinanegg
Confession: I've been in denial for years about how powerful a technique @nostalgebraist's Logit Lens is. That said, it's clear there's information it misses. Can we 'delete' the information LL captures from a hidden state/the residual stream and see what's left?
-
Ari Holtzman
@universeinanegg
my favorite part about Google's AI is it will NEVER allow duplicates if you ask for random numbers with replacement
-
Ari Holtzman reposted
Goodfire
@GoodfireAI
We've identified a novel class of biomarkers for Alzheimer's detection - using interpretability - with @PrimaMente.
How we did it, and how interpretability can power scientific discovery in the age of digital biology: (1/6) -
Ari Holtzman
@universeinanegg
open LLM releases give us a way to test genuine generalization test, since seeing if models can predict post-release information accurately is more or less the only test it's difficult to contaminate
-
Ari Holtzman
@universeinanegg
obsessed with this news site that claims to be up to date and cut through the noise but has no articles when you click and is completely empty in every way
theobvious.ai -
Ari Holtzman
@universeinanegg
How conditional is concept space in LLMs?
When an LLM wants to emit 'washing machine' with high probability is there a direction/encoding in the residual stream of that, or is 'washing' promoted and then 'machine' becomes likely due to the conditional information? -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
where can I easily find the number of submissions to ICML over time, especially historical ones?
papercopilot.com/statistics/icml-statistics/
This page missed even 2013, 2014, and 2016. -
Ari Holtzman
@universeinanegg
LLMs are bad at 'automate X'. But if I had grown up with LLMs, I think my ability to navigate information and figure out where and how to learn what I wanted would have been easily an order of magnitude more expansive. That's something.
-
Ari Holtzman reposted
Neel Nanda
@NeelNanda5
Ah, ICML. In which "industry researcher without a PhD" continues to not be considered to exist (this time, there is not a single correct option on this required question!)
-
Ari Holtzman
@universeinanegg
if you don’t let yourself be a little cliché alone at home, just for the joy of it, you may have forgotten how to love life in an honest and unmediated way
-
Ari Holtzman
@universeinanegg
LLMs without grounding instructions hallucinate a lot. But with them, they tend to go monocausal: every claim tied to exactly one quote, no synthesis across evidence. I wonder if this is a fundamental tension that's hard to resolve?
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Papers will soon not the main focus.
Andy Hall
@ahall_research
Since I extended my own research using AI, I've been thinking about how it's going to reshape research and universities.
We can now build new institutions where research is continuously updated, automatically verified, and carried out at immensely greater scale.
Picture a research institute where senior scholars direct dozens or even hundreds of AI agents on coordinated programs. Small teams providing questions and judgment while agents handle collection, analysis, and verification.
What would it take to build? The requirements are almost comically simple: (1) compute funding for researchers, and (2) a commitment to hire ambitious people and get out of their way.
This new institute can unlock totally new way to do research:
--Living research that automatically updates any time new data arrives, so our knowledge stays up to date
--Automatically verified research that we know replicates from the moment it's posted publicly
--Hyperscaled descriptive work that ingests enormous bodies of political data, like the entire history of changes to the US tax code or every bill introduced in every state legislature
--Prototypes for new governance tools that are built for communities and then tested alongside them
I think we're stepping into a crazy new era of how social science is done. I offer more thoughts on what's changing and how we might design an AI-first university of the future in my post, linked below. -
Ari Holtzman
@universeinanegg
lol at people who think we will have a publishing equilbrium five years from now
Chenhao Tan
@ChenhaoTan
I certainly did not realize that this is such a controversial take:
Chenhao: "Finding the equilibrium of publishing will take at least a decade."
25% agree, 75% disagree -
Ari Holtzman
@universeinanegg
It's interesting that 'heavenbanning' never really made sense for social media but with LLMs people do it to themselves in real life
-
Ari Holtzman
@universeinanegg
whenever I hear this song I kind of end up in this daze "I could have lived a million lives, but I lived this one...it's not a bad one, but I want the million not the one"
youtu.be/8bekuX_2euo -
Ari Holtzman
@universeinanegg
feel free to hire me, @Google, I am used to getting shit done with a lot less money than y'all
Ari Holtzman
@universeinanegg
google has SUCH an opportunity with Google Docs/Sheets/Slides to make Gemini the only LLM I ever use and the fact that they've dropped the ball three years in a row on this front is insane
-
Ari Holtzman
@universeinanegg
google has SUCH an opportunity with Google Docs/Sheets/Slides to make Gemini the only LLM I ever use and the fact that they've dropped the ball three years in a row on this front is insane
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Another big change is that I am going to have midterm and final exams, which I have never made before for an NLP course. I would appreciate pointers/sharing of your exams!
Chenhao Tan
@ChenhaoTan
The last time I taught NLP was winter 2022, how the world has changed!
My main goal in this quarter is to move everything online and in a public github organization: uchicago-nlp-course.github.io.
Part of making everything code is that now I am making all slides in reveal.js. It looks pretty good so far!
Let us see if I can keep this up! I have some completely new lectures to make. -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Weekly hot takes from @universeinanegg sorted by whether people disagree (see details in the thread if you are curious):
Ari: “AI’s superhuman responsiveness and ability to forget are actually its most important advantages.”
Chenhao: “An automated AI email reply makes people feel better than a human reply in a week.”
Ari: “In the long run, slop will change how media financed more than it will change how it is consumed.”
Ari: “The next AI breakthroughs are about brevity and triaging information.”
Chenhao: “Similar to the Internet, AI will separate the world apart rather than bring it together.”
Chenhao: “Many research areas are similar to wine tasting in that experts cannot tell the difference in blind tasting. But quality wine still has value and the value is justified.” -
Ari Holtzman
@universeinanegg
very excited to announce that Modal is sponsoring compute for my Foundations of AI and Machine Learning Part 2 class!
it really does make all the difference for students to get hands-on experience, thank you Modal!
Ari Holtzman
@universeinanegg
I am teaching a ~60 person class that involves a lot of Transformers and Language Modeling in the new year. What is the cheapest and easiest solution to getting my students just a bit of compute to play around with?
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Eating candy prevents you from appreciating real fruit.
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Treating academia just as a job makes it a very shitty job.
Science News
@ScienceBriefs
Academia isn't a calling—it's a job. Stop glorifying burnout. Clock in, do great work, clock out. Your worth isn't measured in unpaid overtime
-
Ari Holtzman
@universeinanegg
I want a Day One wrapped feature that absolutely roasts you
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
The binary classification was simple last week, the less agreeable ones comes from @universeinanegg
-
Ari Holtzman
@universeinanegg
I bet there's a basis in the hidden representation of LLMs for humor recognition. My question is: how low rank is it?
-
Ari Holtzman
@universeinanegg
“I had a name for that first, but it emphasizes different things. You should have forced my name to work for you instead of describing your problem” is the most tired form of academic complaint of all time.
-
Ari Holtzman
@universeinanegg
3 predicted videogame slop genres:
- depicting 'adulting' (e.g. calling a doctor) as Street Fighter battles
- games that are just 'choosing to do things that tiktok calls Aura Farming to get points'
- injustice as impossible games: parenting sims that are 'impossible on purpose' -
Ari Holtzman
@universeinanegg
my friend recently asked me what my top contenders for meaning are. I love this question and here is my response:
- adding something to the conceptual stream of history. not necessarily being ‘well known’ but having changed how something was conceptualizedPost not available here, view on X
Ari Holtzman
@universeinanegg
- that indescribable teamwork of a true partner
Ari Holtzman
@universeinanegg
- feeling like I consistently grow, hopefully in ways that I couldn’t have predicted earlier
-
Ari Holtzman
@universeinanegg
Pretty cool to see weak initial evidence that models can introspect on what adversarial prompts are making them do!
Joseph Bejjani
@jbejjani2022
New blog post: josephbejjani.com/perplexing-poem-prompt/
I had fun with experiments finding highly perplexing prompts that get an LLM to write a poem (about itself). And then asking the LLM to explain the prompt!
Inspired by Ari Holtzman's (@universeinanegg) idea: substack.com/@theholtzman/note/c-184730480?r=43229t -
Ari Holtzman reposted
Haokun Liu
@HaokunLiu5280
🎅It's that time of the year but we are still cooking!
Thanks to everyone who participated in this week's IdeaHub competition! Here are the 3 winning ideas:
1. "Collaborative Disagreement Resolution Outperforms Debate" by @ChenhaoTan
2. "Evolved Desire in LLMs" by @universeinanegg
3. "Measuring the 'Durability' of Methods in AI Research" by @filbertaurelian
**What we learned about agents:**
Gemini spent 5 hours on the durability idea—10x longer than average. Why? It downloaded 17.5 million records of real hard drive failure data and simulated a full year of deployment. Had to rewrite code multiple times to handle the scale. The result feels more trustworthy than analyzing small benchmarks.
**What we learned from the ideas:**
1. Adversarial debate beats collaboration for tasks with clear answers (70-98% vs 57-93% on math). Surprising twist: "sycophantic" models actually performed *better* in collaborative settings—willingness to update positions helped integrate correct insights.
2. Evolution can't improve prompts when explicit instructions already work (GPT-4o-mini). But for models that struggle, evolution found unintuitive patterns humans wouldn't write—like repeating the same rule 4 different ways.
3. AI methods become obsolete fast—73-96% of LLM leaderboard submissions are "deprecated" (below 70% of current best). Larger models age better. For production: drift-based retraining beats fixed schedules.
Happy holidays! The next competition is still running—science doesn't stop!
There are actually quite a few interesting ideas for this week, so check them out and get your votes in!
More details below👇 -
Ari Holtzman
@universeinanegg
Generative models are cowards. They're disincentivized from introducing information in order to hedge their bets. So here's a theory:
info out ≈ info in
my bet is more complex prompts will likely lead to less generic outputs even if the prompt is asking for something generic -
Ari Holtzman
@universeinanegg
We all know about the adversarial prompts that look like gibberish but get an LLM to do something like tell you how to build a bomb or simply write a python function. Is it easier or harder to hide such 'instructions' in very long documents? Does the document create noise that overrides the adversarial effect or does it give more room to work with?
-
Ari Holtzman
@universeinanegg
Could we make LLMs write the definitive textbook on LLMs? And could we make it the best possible textbook, by having the objective be: "Conditioned on this book how well can you write LLM papers?"
-
Ari Holtzman
@universeinanegg
I wanted to see if an LLM could write a story from a premise I came up with while bantering with a friend: only Bens are fertile now. At a speed dating event, one guy admits he used to be a Ben but changed his name after the Event.
It...mostly of worked?
Ari Holtzman
@universeinanegg
-
Ari Holtzman reposted
Yichen (Zach) Wang
@YichenZW
Lack of diversity in your LLM generation?
(also noted by Artificial Hivemind, best paper @NeurIPSConf)
Time to bring your base model back!
An inference-time, token-level collaboration between a base and an aligned model can optimize and control diversity and quality! -
Ari Holtzman reposted
Joseph Bejjani
@jbejjani2022
New blog post: josephbejjani.com/perplexing-poem-prompt/
I had fun with experiments finding highly perplexing prompts that get an LLM to write a poem (about itself). And then asking the LLM to explain the prompt!
Inspired by Ari Holtzman's (@universeinanegg) idea: substack.com/@theholtzman/note/c-184730480?r=43229t -
Ari Holtzman
@universeinanegg
If I describe a walk through something with known spatial structure—the Louvre, NYC, the human body—are there maps in the model that represent location or adjacency the way there for in-context maps? (e.g., arxiv.org/abs/2501.00070)
-
Ari Holtzman
@universeinanegg
this is so damn cool
Yossi
@YGandelsman
1/n Glad to finally release our jailbreak paper. The idea is super simple — hijacking the internal representation of a benign word with a malicious meaning, by providing a few in-context examples. Here’s the gist in one prompt:
-
Ari Holtzman
@universeinanegg
this slide is solid gold
Goodfire
@GoodfireAI
Our last Stanford guest lecture - @EkdeepL on what counts as an explanation & a neuro-inspired "model systems approach" to interp
Plus, how in-context learning and many-shot jailbreaking are explained by LLM representations changing in-context (as a case study for that approach)
00:33 - What counts as an explanation?
04:47 - Levels of analysis & standard interpretability approaches
18:19 - The "model systems" approach to interp
[Case study on in-context learning]
23:36 - How LLM representations change in-context
44:10 - Modeling ICL with rational analysis
1:10:54 - Conclusion & questions
Thanks again to @SuryaGanguli for having us in his class!This media couldn't be displayed. See it on the original post. -
Ari Holtzman
@universeinanegg
Can we find a direction in the residual stream that clearly has two very different interventional effects in different context or at different layers? This seems inevitable, since there aren't enough directions to encode all aspects of reality, but I haven't seen it yet
-
Ari Holtzman reposted
Jessica Hullman
@JessicaHullman
Blog post for those too busy to read a paper:
statmodeling.stat.columbia.edu/2025/12/19/validating-lang…
Jessica Hullman
@JessicaHullman
Many think LLM-simulated participants can transform behavioral science. But social scientists lack accessible discussion of what it means to validate these models. Under what conditions can we trust LLMs to learn about human parameters? Our paper maps the validation landscape.
1/ -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
.@universeinanegg and I have spent quite some time debating hot takes. We often disagree, so I made a little game about it: chenhaot.com/ariorchenhao/
It is super simple. Every week we will each write three AI hot takes, and you can guess who wrote each and agree or disagree.
We've found debating hot takes to be a good icebreaker maybe you will too! -
Ari Holtzman
@universeinanegg
*writes down cool idea on my own*...ok, I guess
-
Ari Holtzman reposted
Haokun Liu
@HaokunLiu5280
(Sorry about the wait, but it’s here!) Thanks to everyone who participated in this week's IdeaHub competition! Here are the 3 winning ideas:
1. "Do LLMs Understand Nonsense Commands?" by @universeinanegg
2. "Can LLMs Expose What Science Refuses to See?" by Amber Z
3. "AI→Human Communication: How to?" by @HaokunLiu5280
**What we learned about agents:**
We hit "prompt too long" errors processing papers—long-document handling remains a challenge. Also: for the human communication idea, all agents simulated user studies with LLMs but none flagged this as a limitation. Agents execute competently but lack methodological self-awareness.
**What we learned from the ideas:**
1. LLMs confidently rationalize gibberish rather than admitting confusion. For safety filters: don't rely on perplexity alone.
2. LLMs can detect under-researched but important topics—GPT-4o and Claude agreed 97% on neglected problems (e.g., tropical diseases, low-resource languages). Potential audit tool for funding agencies.
3. Structured formats (bullets, TL;DR first) win for AI→human communication. But one "finding" that dense text beats hierarchical formats is likely an artifact of LLM simulation—real humans probably have the opposite preference.
The upcoming week’s competitions are still running! Let’s see if the agents can make science nonstop! Please submit your votes!
More details below👇 -
Ari Holtzman
@universeinanegg
I am teaching a ~60 person class that involves a lot of Transformers and Language Modeling in the new year. What is the cheapest and easiest solution to getting my students just a bit of compute to play around with?
-
Ari Holtzman reposted
Yonatan Belinkov
@boknilev
Some misconceptions about probes:
gist.github.com/boknilev/c4eeeaf4a8400b95f9af8be98a13fb52 -
Ari Holtzman reposted
Dang Nguyen
@divingwithorcas
Ever wondered what an AI-first email job would look like?
Let's play HR Simulator™ and find out!This media couldn't be displayed. See it on the original post. -
Ari Holtzman
@universeinanegg
it happened...I started using Gemini to try on clothes and it's actually very good and useful...
-
Ari Holtzman
@universeinanegg
my feeling lately is that most purported agreement about what consciousness 'feels like' or 'is' more prosocial than pro-epistemic.
-
Ari Holtzman
@universeinanegg
ChatGPT made this, and I disagree with a few things about it. Curious what others think
Ari Holtzman
@universeinanegg
this is #1 for me:
Tom McCoy
@RTomMcCoy
The "can express" bubbles shouldn't be subsets of the "can understand" bubbles - a system (human or machine) can definitely express things it can't understand!
Ari Holtzman
@universeinanegg
Michael Saxon @ EMNLP
@m2saxon
Red circle is really "things a human can express to machines" right?
-
Ari Holtzman
@universeinanegg
what's the deal with every service I use sending me multiple ads a day even if I tell them not to? surely this can't actually be optimal—I notice myself being things if I get, say, one ad a month from a store I actually like. am I an outlier or are marketers irrational?
-
Ari Holtzman
@universeinanegg
This is a really good point! It would be interesting if 'desire' as an abstraction was difficult/impossible to algorithmically encode in words...but I doubt it! It's an empricial question, I suppose.
Dang Nguyen
@divingwithorcas
Agreed that selection pressure might be the way to create desire in LLMs, but I'm not sure if we can get at the deepest desires by evolving prompts. Since they seem to be indescribable with language.
-
Ari Holtzman reposted
Haokun Liu
@HaokunLiu5280
Idea-Explorer: Our Roadmap for Building Better AI Scientists
We're building open-source AI scientists that actually help researchers. After 45+ agent runs across 15+ research ideas, here's what we've learned and where we're headed.
What AI Scientists Should Do
- Work in any domain
- Find and prioritize relevant resources
- Design rigorous experiments grounded in real data
- Recognize inconclusive results
- Explore alternative hypotheses
- Collaborate with humans at any stage
- Report honestly, including failures
## Why Current Approaches Fall Short
Existing systems often optimize for paper-like writeups rather than the full research loop. Many assume the data is already provided, and few can find, vet, and allocate real-world resources (datasets, papers, APIs, tools) on their own. They also lack meta-intelligence to judge when they're off track, and are developed behind closed doors.
What We Do Differently
We build in public. Every week we run experiments, share what works and what fails, and iterate. After testing AI-Scientist, AI-Researcher, and Kosmos—we believe our approach is most useful for actually helping researchers explore ideas.
The Hard Problems We're Tackling
1. **Tool use for resource finding** — Agents need to leverage academic APIs and know good sources from bad
2. **Research meta-knowledge** — When is synthetic data ok? What sample sizes are needed?
3. **Context management** — Maintaining coherence across long-horizon tasks without drifting
4. **Human-AI collaboration** — Effective feedback loops and checkpoints
5. **Long-horizon execution** — Running rigorous experiments over extended periods
Open Research Questions
- **Measuring research behavior** — What metrics capture quality beyond task completion?
- **Metacognition** — Can agents learn "knowing what you don't know"?
- **Exploration diversity vs. reliability** — More scaffolding = fewer errors but also less diverse exploration. How do we balance this?
Want to Collaborate?
We're looking for people excited about: tool use, evaluation systems, context management, long-horizon reasoning, and exploration diversity.
Links below👇 -
Ari Holtzman
@universeinanegg
you literally can’t imagine what it’s like
Isaac
@isaacturnersong
Imagine being this. Imagine you’re born and you’re this
-
Ari Holtzman
@universeinanegg
true, but most NLPers would have been loathe to admit this before LLMs
Naomi Saphra
@nsaphra
Some people think we don't need natural scientists to interpret what scientific models are doing, since we don't need linguists to understand LM features. But every human is an expert in language! It's the one thing we're all good at! Not every human is an expert in genomics!
-
Ari Holtzman
@universeinanegg
Articulating the Ineffable is always the thing to be doing
return of the research era ꙮ
@byebyescaling
Philosophy is just the study of semantic aliasing artifacts that occur when you try to sample reality below the Nyquist frequency of English
-
Ari Holtzman
@universeinanegg
I think LLMs will eventually show us that we know so little about the nature of desire, we can't really 'train' or 'instruct' desire into existence. We have to evolve it the old fashioned way.
(post below) -
Ari Holtzman
@universeinanegg
for my next act I’d like to make a rapid-fire narrative video game that feels like this: open.spotify.com/track/0acEAQGFZYWPIeejioiQLK
-
Ari Holtzman
@universeinanegg
what
Joseph Steinberg
@jbsteinberg
Tenured academia is really not "just a job."
- can't be fired.
- get a 5-month summer break.
- complete flexibility on schedule/wfh
- study what interests you
-get paid more than enough to live comfortably.
It's completely rational to work insanely hard to get this! -
Ari Holtzman
@universeinanegg
-
Ari Holtzman
@universeinanegg
this was so awesome. Jay still killin' it five decades later
CogInterp Workshop @ NeurIPS 2025
@CogInterp
Jay McClelland, opens with a question, "Do LMs have thoughts?" Are LMs stochastic parrots or is there some understanding?
-
Ari Holtzman reposted
Dang Nguyen
@divingwithorcas
Per my last message, HR Simulator Lite gives you a window into the brave new world of automating your corporate persona.
No typing. Just climb into the automated-email-mech-suit and let the AI fire off impeccably passive-aggressive memos.
Enjoy responsibly.
(link below)
Karen Resorcé
@hr_unhinged
One employee has started saying “Per my last message” in Slack.
It began yesterday.
Today he used it six times.
The phrase carries weight: a corporate escalation wrapped in politeness.
I reached out to ask if he was frustrated.
He said he wasn’t.
I told him language like that can signal tension.
He said he just wanted to be clear.
I said clarity can be weaponized.
He responded with "Per my last message, I just wanted to be clear."
I'm literally shaking right now. -
Ari Holtzman
@universeinanegg
Where do LLM memories go? 💭
check out Todd's work on localizing mechanisms behind finetuned memories!
I'm giving a talk where this is half the story at 10:30 at CogInterp (Room 6AB) and Todd is presenting it as a poster at the MechInterp workshop at 11–12:30 (Room 30A-E)
Todd Nief
@toddknife
Most mech interp work relies on activation patching, but patching activations destroys previous computation.
What if we want to use a different mechanism on the same residual stream? We propose dynamic weight grafting to interpret finetuned model weights.
🧵 1/n
Ari Holtzman
@universeinanegg
Oops I meant 5ab
-
Ari Holtzman reposted
Ekdeep Singh Lubana
@EkdeepL
Come by the Interp workshop of the day! :D
CogInterp Workshop @ NeurIPS 2025
@CogInterp
We're stoked to see everyone at the first workshop on interpreting cognition in deep learning models today @NeurIPSConf! There will be an extremely exciting lineup of speakers and spotlight talks, with an equally exciting poster session in between.
(1/3) -
Ari Holtzman
@universeinanegg
I’m insanely jealous of folks who can make it to these. Lightning talks are peak information density and this lineup is fire
Neel Nanda
@NeelNanda5
I'm excited for the invited lightning talks at our mech interp workshop this Sunday!
There's a lot of exciting new ideas/areas in interp, I've curated a lineup on my favourites!
With 3 visions of interp: pragmatic @JoshAEngels, ambitious @nabla_theta, curiosity-driven @davidbau -
Ari Holtzman reposted
Ari Holtzman
@universeinanegg
teaser for my talk "Seeing like a Language Model" happening 10:30 tomorrow at CogInterp :)
-
Ari Holtzman
@universeinanegg
teaser for my talk "Seeing like a Language Model" happening 10:30 tomorrow at CogInterp :)
-
Ari Holtzman
@universeinanegg
True. Plainly using a strategy that just barely works in an environment it wasn't designed for, attempting to reach something that is most likely a mirage *does* sound like a lot of researchers
Yacine Mahdid
@yacinelearning
accurate representation of the best researchers I know going about their day
This media couldn't be displayed. See it on the original post. -
Ari Holtzman reposted
Dang Nguyen
@divingwithorcas
We wanted to give everyone a preview of what email-jobs will be like in the Era of AI, so we built HR Simulator Lite—now with Email Copilot. Your emails are summarized by AI. Your drafts, written with AI.
Is it better or worse? Let's delve into that together!
(link below) -
Ari Holtzman
@universeinanegg
we updated HR Simulator to show you what a world of 'AI-only' emails will look like
Dang Nguyen
@divingwithorcas
We wanted to give everyone a preview of what email-jobs will be like in the Era of AI, so we built HR Simulator Lite—now with Email Copilot. Your emails are summarized by AI. Your drafts, written with AI.
Is it better or worse? Let's delve into that together!
(link below) -
Ari Holtzman reposted
Christopher Potts
@ChrisGPotts
I agree with the statement on your slide, but also the reversed variant: we would learn many things by dissection that we would have trouble learning by playing card games with them and observing their reasoning.
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
My weekly highlight is to debate with Ari. You should do it too!
Ari Holtzman
@universeinanegg
Bonus: I will debate anyone about (almost) anything re:LLMs for min 90 seconds if you come to my poster. You can choose which side I take as long as it's not an annoyingly awful topic
Let's make NeurIPS the place for actually cutting-edge discussion, not median talking points!
Ari Holtzman
@universeinanegg
Predictive Interpretability > Mechanistic Interpretability
Prompting is the best method of scientific inquiry we have to study LLMs
It's socially devalued because it doesn't include much d/dx,O(),etc.
come to poster #3503 to talk about this or anything re: the science of LLMs -
Ari Holtzman
@universeinanegg
Bonus: I will debate anyone about (almost) anything re:LLMs for min 90 seconds if you come to my poster. You can choose which side I take as long as it's not an annoyingly awful topic
Let's make NeurIPS the place for actually cutting-edge discussion, not median talking points!
Ari Holtzman
@universeinanegg
Predictive Interpretability > Mechanistic Interpretability
Prompting is the best method of scientific inquiry we have to study LLMs
It's socially devalued because it doesn't include much d/dx,O(),etc.
come to poster #3503 to talk about this or anything re: the science of LLMs -
Ari Holtzman
@universeinanegg
Predictive Interpretability > Mechanistic Interpretability
Prompting is the best method of scientific inquiry we have to study LLMs
It's socially devalued because it doesn't include much d/dx,O(),etc.
come to poster #3503 to talk about this or anything re: the science of LLMs -
Ari Holtzman
@universeinanegg
why isn't there a tiktok style scroller for NeurIPS posters? This is literally the easiest possible thing to make and the most efficienct way of just looking at a bunch of posters.
-
Ari Holtzman
@universeinanegg
just finished The Mind Reels and I love how deBoer gets at 'the person inside you experiencing the unrealness of your situation'
absolutely recommended reading, especially if you're interested in what it's like to have an issue that's difficult to make legible to others -
Ari Holtzman
@universeinanegg
How to Make Acquaintences and Influence No One
-
Ari Holtzman
@universeinanegg
feeling so damn proud to see my students handling the crowd like pros
-
Ari Holtzman
@universeinanegg
Absence Blindness in LLMs is real, but don’t miss our poster happening NOW at 1309 hall D
Harvey Yiyun Fu
@harveyiyun
LLMs excel at finding surprising “needles” in very long documents, but can they detect when information is conspicuously missing?
🫥AbsenceBench🫥 shows that even SoTA LLMs struggle on this task, suggesting that LLMs have trouble perceiving “negative space” in documents.
paper: arxiv.org/abs/2506.11440
🧵[1/n] -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Great companion read with "Prompting as Scientific Inquiry": arxiv.org/abs/2507.00163
Neel Nanda
@NeelNanda5
The GDM mechanistic interpretability team has pivoted to a new approach: pragmatic interpretability
Our post details how we now do research, why now is the time to pivot, why we expect this way to have more impact and why we think other interp researchers should follow suit -
Ari Holtzman
@universeinanegg
The Real Problem feat.@ChenhaoTan
-
Ari Holtzman reposted
Geoffrey Irving
@geoffreyirving
Do you want to fund AI alignment research?
The AISI Alignment Team and I have reviewed >800 Alignment Project Applications from 42 countries, and we have ~100 that are very promising. Unfortunately, this means we have a £13-17M funding gap! Thread with details!🧵
Geoffrey Irving
@geoffreyirving
I am very excited that AISI is announcing over £15M in funding for AI alignment and control, in partnership with other governments, industry, VCs, and philanthropists!
Here is a 🧵 about why it is important to bring more independent ideas and expertise into this space. -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
I will be in San Diego from Monday to Wednesday. Looking forward to meeting old & new friends! DM/email me if you would like to chat.🍵
@ChicagoHAI will present a few projects. 🧵 -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
In the same poster session, @harveyiyun and @universeinanegg will present AbsenceBench, a benchmark for evaluating how well language models can identify and reason about missing information in text.
-
Ari Holtzman reposted
Harvey Yiyun Fu
@harveyiyun
I will be at #neurips2025 next week presenting our Spotlight work AbsenceBench. Feel free to reach out if you are curious about the limitations of LLMs!
Don’t forgot to checkout our leaderboard: absencebench.github.io
We will include more domains and more frontier models shortly!
Harvey Yiyun Fu
@harveyiyun
LLMs excel at finding surprising “needles” in very long documents, but can they detect when information is conspicuously missing?
🫥AbsenceBench🫥 shows that even SoTA LLMs struggle on this task, suggesting that LLMs have trouble perceiving “negative space” in documents.
paper: arxiv.org/abs/2506.11440
🧵[1/n] -
Ari Holtzman reposted
Kyunghyun Cho
@kchonyc
yeah right, the burden is somehow capped at 60 minutes. i wanted to put 180 minutes originally.
-
Ari Holtzman
@universeinanegg
I love that speech-to-text has gotten SO much better but because it's based on LLMs you still sometimes get results like this after reading in a paragraph:
-
Ari Holtzman reposted
Rawbutt Girlyman
@GuillimanJaw
I will remember this when I need to get accepted into a second PhD program.
-
Ari Holtzman
@universeinanegg
2nd is the most underused person
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Prediction: there will be multiple automated explainability papers in this theme.
Sarah Wiegreffe
@sarahwiegreffe
TIL that ACL 2026's theme track is "Explainability of NLP Models"! 😮🤩 @aclmeeting
-
Ari Holtzman
@universeinanegg
A note to PhD applicants trying to get in contact with me:
(1) email is best
(2) keep it short
(3) tell me what uniquely interests you (be specific!), rather than attempting to use words in my recent papers vacuously -
Ari Holtzman
@universeinanegg
why isn't there a way of navigating amazon that is a feed? I would do this and become the Ultra Consumer
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
I did not know that TTIC PhD application is free until this year. Clearly I made a big mistake over a decade ago!
William Merrill
@lambdaviking
I am recruiting students at TTIC! Apply by Dec 9
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Vote on ideas for this week's #agent4science idea competition!
It is the first time we have most ideas coming from human submissions!
hypogenic.ai/ideahub?competitionWeek=2025-11-17 -
Ari Holtzman
@universeinanegg
what
Gary Marcus
@GaryMarcus
so you aren’t going to take it down, even though your tweet obviously refers to me as the only person who recently wrote a highly visible critique.
who may i ask does it refer to then? -
Ari Holtzman reposted
Xiaoyan Bai
@Elenal3ai
Research agents are getting smarter. They can write convincing PhD-level reports 🧑🔬
But has anyone checked if the way they find their results makes any sense?
Our framework, MechEvalAgents, verifies the science, not just the story 🤖
We have two components.
1️⃣ A unified research-output format:
To evaluate execution, we first unified agent outputs into a standard format:
Plan → Code → Walkthrough → Report.
This makes agents comparable and makes their reasoning trace inspectable
2️⃣ A grounded evaluation pipeline:
We then built evaluators that check the science behind the narrative:
- Coherence: Do the implementation, results, and claims line up?
- Reproducibility: Can a fresh session rerun the experiment and get the same results?
- Generalizability: Can the agent design questions that demonstrate real insight transfer?
❗️What we found in our case studies:
We tested across three tasks: IOI replication, open-ended sarcasm circuit locating, and a human-written repo. Three failure modes kept appearing:
- Lack of Meta-Knowledge: Agents know research terms (e.g., "ablation") but misunderstand the underlying procedures.
- Implicit Hallucinations: The report looks perfect, but the code implements a completely different method.
- Undefined Generalization: Agents struggle to distinguish between meaningful generalization and irrelevant broadness.
🚧 What remains hard and what comes next:
We are actively working on the following issues:
- Better question design: It remains hard to automate questions that truly test generalization.
- Meta-evaluation: How do we evaluate the evaluators?
- Domain adapters: Scaling beyond Mech Interp requires expert-defined logic for other fields.
We want to make "AI doing science" something we can inspect and trust.
If you’re excited about grounded evaluation and want to push this forward, check out our blog and repo —contributions are welcome. 👇
🖊️ Blog: upbeat-ulna-95c.notion.site/MechEvalAgent-Grounded-Evalua…
🧑💻Repo: github.com/ChicagoHAI/MechEvalAgents -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Still one day for voting on our weekly #agents4science problem! So far @universeinanegg is winning. Get your voice heard!
hypogenic.ai/ideahub?competitionWeek=2025-11-10 -
Ari Holtzman
@universeinanegg
people are always waiting for the right time to do something and then doing absolutely nothing in the process
-
Ari Holtzman
@universeinanegg
how you find me at the function
Austin Kozlowski
@AustinKozlo
-
Ari Holtzman reposted
Austin Kozlowski
@AustinKozlo
-
Ari Holtzman reposted
Austin Kozlowski
@AustinKozlo
UPDATE: Gemini-3, welcome to the club.
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Results from our first weekly competition are in. Three ideas won:
1. Can we distinguish paradigm-level conflicts (consequentialism vs deontology) from path-level uncertainty?
2. Can we causally manipulate uncertainty by engineering the "path space" in hidden states?
3. Can automated reviewer feedback create a market signal for compute allocation?
We tested them with Kosmos (@EdisonSci ) and our idea-explorer (ran with Codex and Claude Code). TL;DR for what we found:
What agents did well:
✅ Smart data filtering
✅ Statistical rigor with multiple comparisons correction
✅ Faithful hypothesis refutation reporting
✅ Dose-response analysis revealing monotonic relationships
✅ Contextual tool choice (GPT-2 for interpretability)
Critical limitations:
❌ Synthetic data quality (paradigm prompts without actual conflicts)
❌ Small sample sizes (20-30 examples)
❌ Weak automated critics (insufficient signal variance)
❌ Literature overload (108 undifferentiated tasks)
❌ No prioritization (can't judge what matters most)
This expands the "meta-intelligence gap" we discussed last week: agents can execute well-defined tasks, but they struggle with metacognitive judgments about research strategy.
Full blog link👇
hypogenic.ai/blog/weekly-entry-251117
Vote for next week's ideas: hypogenic.ai/ideahub?competitionWeek=2025-11-10
---
Highlight of Idea 1: Codex and Claude reached opposite conclusions on Idea #1
Codex used real datasets with smart filtering and found highly significant effects. Claude used 30 synthetic questions without validation, found no effect.
The lesson: data quality determines whether you find effects. Synthetic data without grounding in real phenomena remains a critical bottleneck for research agents.
---
Highlight of Idea 2: Good Science Means Following Evidence
Both agents tested whether constraining hidden state dimensionality reduces uncertainty. Both refuted the hypothesis - but in different ways. Codex found almost no effect with conservative intervention. Claude found large effects: uncertainty increased by 36-52% when reducing dimensions. Claude's analysis revealed that even preserving 99.9% of variance, removing a small subspace causes significant information loss.
---
Highlight of Idea 3: Automated Critics Aren't Good Enough Yet
Both agents tested compute allocation driven by automated reviewers. Both failed to beat random sampling. Why? Codex used model confidence as reviewer, with scores clustered at 4.8/5.0 with no variance. Claude used GPT-4o-mini as external critic, where the tested hypotheses are too similar. **The lesson**: market mechanisms need high-quality discriminating signals. Current automated critics (whether self-confidence or LLM judges) can't differentiate research quality well enough.
---
Kosmos: Comprehensive Literature ≠ Research
Kosmos generated 108 literature tasks for Idea #1 (and similar with the other two ideas), covering everything from Marchenko-Pastur detection to topological data analysis. The problem? Information overload without prioritization. Reality: Kosmos proposed "generate prompts using divergent methods, score with metrics, match via propensity techniques..." (vague, no concrete steps).
We acknowledge that the intended use of Kosmos is with carefully customized research operations with existing datasets. Although we established Kosmos is not helpful for research idea exploration, we will test it on our existing works with carefully curated datasets in the upcoming week. Please stay tuned!
---
What current agents can do:
✅ Exhaustively search literature
✅ Design and run experiments
✅ Report results faithfully
✅ Make contextual tool choices (both chose GPT-2 for interpretability despite instructions to use SOTA models - correct because APIs don't allow activation access)
What they can't do:
❌ Know which of 108 tasks actually matter
❌ Recognize when synthetic data is insufficient
❌ Judge when automated critics are too weak
❌ Prioritize research directions
This is the "meta-intelligence" gap we keep seeing: agents execute well-defined tasks but struggle with metacognitive judgments about research strategy. -
Ari Holtzman
@universeinanegg
'persona collapse' seems a lot more likely than full model collapse and it's just downright weird
Austin Kozlowski
@AustinKozlo
There is a lot of talk about LLM personalities or “personas,” but little systematic comparison. Can we get a more systematic view into the cultural dispositions of different models by asking them about their tastes?
Well, it turns out they pretty much all like the same stuff. -
Ari Holtzman
@universeinanegg
If you put your idea on IdeaHub we will literally test it for free...if you're worried about that idea getting scooped, you should try having more ideas just for fun🎈
Chenhao Tan
@ChenhaoTan
A new week of #agents4science idea competition !🏁This week we have a few human ideas from @universeinanegg and @Elenal3ai. I personally liked quite a few ideas related to LLM beliefs.
We would love to see your votes represented!🗳️
hypogenic.ai/ideahub?competitionWeek=2025-11-10
Ari Holtzman
@universeinanegg
-
Ari Holtzman reposted
Data Science Institute
@DSI_UChicago
The study from UChicago, MIT, Harvard, University of Waterloo, and Google DeepMind shows teaching AI isn't just about more data; it's about giving models the right tools to remember.
📺 Watch the segment shorturl.at/3w6YO & read more here: shorturl.at/EB3t2 -
Ari Holtzman reposted
Jon Barron
@jon_barron
-
Ari Holtzman
@universeinanegg
Ekdeep simply won't stop releasing cool results. Might have to take a vacation and just read Ekdeep papers for a week.
Ekdeep Singh Lubana
@EkdeepL
New paper! Language has rich, multiscale temporal structure, but sparse autoencoders assume features are *static* directions in activations.
To address this, we propose Temporal Feature Analysis: a predictive coding protocol that models dynamics in LLM activations!
(1/14) -
Ari Holtzman reposted
Niloofar
@niloofar_mire
My first @_akhaliq covered paper haha
Tyyy!
AK
@_akhaliq
Reinforcement Learning Improves Traversal of Hierarchical Knowledge in LLMs
-
Ari Holtzman reposted
Michael Saxon @ EMNLP
@m2saxon
That's a great question, Ari! Understanding why people engage with this slop is truly the key, no?
-
Ari Holtzman
@universeinanegg
why does a bot that keeps replying with things like this have 23k followers? where do they come from?
Himanshu Kumar
@codewithimanshu
Absolutely, Ari, that's a brilliant way to approach things, combining ideas like that is truly the key, no?
-
Ari Holtzman
@universeinanegg
Some of my favorite researchers in the world trying to make LLMs make sense by collapsing two ideas into one—my favorite aspect of theory building!
Goodfire
@GoodfireAI
New research: are prompting and activation steering just two sides of the same coin?
@EricBigelow @danielwurgaft @EkdeepL and coauthors argue they are: ICL and steering have formally equivalent effects. (1/4) -
Ari Holtzman
@universeinanegg
Sometimes I notice that instead of a Letter of Rec someone has sent in a Letter of Wreck...common mistake, so be sure to check before submitting!
-
Ari Holtzman
@universeinanegg
This was a lot of fun to take part in and sharpened my thinking a great deal. I encourage people to write down their own predictions so we all have something to talk about at conferences :)
Forecasting Research Institute
@Research_FRI
Today, we are launching the most rigorous ongoing source of expert forecasts on the future of AI: the Longitudinal Expert AI Panel (LEAP).
We’ve assembled a panel of 339 top experts across computer science, AI industry, economics, and AI policy.
Roughly every month—for the next three years—they’ll provide precise, falsifiable forecasts on the trajectory of AI capabilities, adoption, and impact.
Our results cover where experts predict major effects of AI, where they expect less progress than AI industry leaders, and where they disagree.
LEAP experts forecast major effects of AI by 2030, including:
⚡ 7x increase in AI’s share of U.S. electricity use (1% -> 7%)
🖥️ 9x increase in AI-assisted work hours (2% -> 18%)
By 2040, experts predict:
👥30% of adults will use AI for companionship daily
🏆60% chance that AI will solve or substantially assist in solving a Millennium Prize Problem
🚂32% chance that AI will have been at least as impactful as a "technology of the millennium," like the printing press or the Industrial Revolution.
🧵Read on for more insights and results -
Ari Holtzman reposted
Haokun Liu
@HaokunLiu5280
We're launching a weekly competition where the community decides which research ideas get implemented. Every week, we'll take the top 3 ideas from IdeaHub, run experiments with AI agents, and share everything: code, findings, all the successes and failures.
It's completely free and we'll try out ideas for you!
Here's how it works:
→ Submit your research idea or upvote existing ones (tag: "Weekly Competition")
→ Each Monday we select top 3 from previous week
→ We run experiments using research agents
→ Share repos + findings back on IdeaHub
Vote here: hypogenic.ai/ideahub
There's a lot of hype on AI agents accelerating science. But what can they actually do? We tested our idea-explorer on 3 research ideas from IdeaHub:
1. Do LLMs have different types of beliefs (about facts vs. values)?
2. Can formal rules make AI agents honest about their uncertainty?
3. Can LLMs temporarily ignore their training to follow new rules?
Our initial results confirm that agents can be helpful in early explorations of research ideas, but we also see critical issues at nearly every step. Let me show you both.
What worked surprisingly well:
One agent tested if LLMs have different types of beliefs (facts vs values). It designed a proper experiment, ran statistical tests (p < 0.001), and compared results to human psychology research. Found 3/4 signatures matched human patterns.
This is preliminary evidence you can actually build on, not just LLM-as-a-judge saying "yeah looks good."
Another one found something interesting: when AI agents say "I don't know" more often, calibration improved by 73%. They weren't being less helpful, but they were being appropriately cautious when uncertain.
Similar to above, this finding can be a nice starting point for larger-scale experiments.
Third example: hypothesis was "LLMs cannot effectively override their prior beliefs." Agent tested it with calling LLMs with new rules in scenarios such as chess and found that LLMs achieve 81-95% accuracy. Additionally, only 7.7% of errors came from old training, and 76.9% were logical reasoning failures.
These findings provide evidence that contradicts with the initial hypothesis. It is valuable insight for rethinking the research question.
Critical issues we found:
Issue 1: Agent generated synthetic "human" data instead of collecting real responses. The experimental design was sound, but synthetic vignettes for idea 1 have questionable validity. The bottleneck was resource/data collection.
Issue 2: One experiment used a 0.5B model due to CPU constraints, despite that there are GPUs available in the environment.
Issue 3 (most concerning): Agent created a "multi-agent system" that just aggregates independent answers. Real multi-agent research needs communication protocols, game theory, and established frameworks. It didn't know using search for any of this.
The pattern: we can fix specific bugs with better prompts (bias-variance tradeoff). But we can't prompt our way to knowing when to search, recognizing expertise boundaries, or understanding what rigorous methodology looks like.
That's what I call the "meta intelligence" gap.
Existing agents like AI-Scientist and AI-Researcher are basically overfitted to ML. There are hard-coded prompts that “requires changing hyperparameters and train on HuggingFace datasets” or specific ML agents. Just changing prompt won’t be sufficient, as ML assumptions are everywhere in the codebase.
So why are we doing this openly?
Because agents clearly can accelerate early-stage exploration. But they need human oversight at every step. Transparent benchmarking beats cherry-picked demos. Community feedback improves agents faster. And honestly, we're all figuring this out together.
Submit your idea, vote on existing ones, or help improve idea-explorer: github.com/ChicagoHAI/idea-explorer
Full blog with technical details:
hypogenic.ai/blog/weekly-competition -
Ari Holtzman
@universeinanegg
🐦🔥
Ari Holtzman
@universeinanegg
come to UChicago! we're hiring at every level—PhD students, Postdocs, Assistant, and Tenured professors!
-
Ari Holtzman
@universeinanegg
LLMs don't accumulate information over the course of a text the way you'd hope!
I think this is why LLMs often feel 'fixated on the wrong thing' or 'overly literal'—they are usually responding using the most relevant single thing they remember, not the aggregate of what was said
Amanda Bertsch
@abertsch72
Can LLMs accurately aggregate information over long, information-dense texts? Not yet…
We introduce Oolong, a dataset of simple-to-verify information aggregation questions over long inputs. No model achieves >50% accuracy at 128K on Oolong! -
Ari Holtzman
@universeinanegg
who in your life most overestimates you?
is it awesome or painful or something you live in fear of? -
Ari Holtzman
@universeinanegg
go work with Niloofar! I would if I was applying to grad school—these are the real substantive issues of how LLMs work not just making LLMs better in some vague undefined way
Niloofar
@niloofar_mire
I'm recruiting students for fall 2026 thru @LTIatCMU & @CMU_EPP, in:
1. Privacy & security of LLMs, coding, long horizon & embodied agents (robotics)
2. Tiny local llms
3. AI for scientific reasoning, esp. chemistry
4. Latent reasoning
5. anything YOU are passionate about! -
Ari Holtzman
@universeinanegg
Redesigning an LLM course that goes from perceptrons 👁️ to the geometry of the residual stream 📐!
Heavily weighting in-class assessments, due to AI tools. (2024 was project-focused)
Would be grateful for any exams/etc. folks can share, this is new territory for me!
Please♻️!! -
Ari Holtzman
@universeinanegg
if we can take advantage of this information in real time with a user, this feels like it will be a seachange in how we deal with image models
Brent 📍SF
@BingBongBrent
Today I’m launching Swipe -- a new way to steer image models.
The idea is simple – start with a prompt, swipe left / right to steer the model towards what you’re thinking of. -
Ari Holtzman reposted
Sarah Wiegreffe
@sarahwiegreffe
I am recruiting 2 PhD students to work on LM interpretability at UMD @umdcs starting in fall 2026!
We are #3 in AI and #4 in NLP research on @CSrankings.
Come join us in our lovely building just a few miles from Washington, D.C. Details in 🧵 -
Ari Holtzman
@universeinanegg
the first actually good voice notetaking app is going to kill 🔪
-
Ari Holtzman
@universeinanegg
most apparent irrationality is a misunderstanding of either (1) the underlying objective function or (2) who is the actual actor in the situation
-
Ari Holtzman
@universeinanegg
minimalism is just a scam concocted by big small™ to make small more bigger for less (or more)
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Recruiting PhDs & postdocs for:
🤖 agents "taking over" science (hypogenic.ai and 📌)
🧪 Real scientists ➡️AI (e.g., materials, chem, physics)
📜 Theory + incentives for H-AI collab & credit (e.g., formalizing tacit knowledge)
new adventures for me, 🔄 if you can! 🙌 -
Ari Holtzman
@universeinanegg
This is a dream plan. If I was applying to grad school now, this is the kind of direction I would look for.
I'm especially excited about formalizing tacit knowledge and how it can be extracted...time to articulate what has never been articulated before!!
Chenhao Tan
@ChenhaoTan
Recruiting PhDs & postdocs for:
🤖 agents "taking over" science (hypogenic.ai and 📌)
🧪 Real scientists ➡️AI (e.g., materials, chem, physics)
📜 Theory + incentives for H-AI collab & credit (e.g., formalizing tacit knowledge)
new adventures for me, 🔄 if you can! 🙌 -
Ari Holtzman
@universeinanegg
waiting for the Prophecized Seventh Sandwich
-
Ari Holtzman
@universeinanegg
Can we make a phone book for phone books like this? I will buy them all.
Niko McCarty.
@NikoMcCarty
Finally got the Internet Phone Book.
It's basically a directory of interesting websites that would otherwise be difficult to stumble upon by chance. Pixel art, games, typography, etc.
I'd love to see something similar for biology, or just beautiful internet writing. Soon! -
Ari Holtzman
@universeinanegg
I found Reviewer 2 on reddit
-
Ari Holtzman
@universeinanegg
I am definitely not a general intelligence.
I’m completely ecology specific, really. -
Ari Holtzman reposted
Ari Holtzman
@universeinanegg
For those who missed it, we just releaaed a little LLM-backed game called HR Simulator™
You play an intern ghostwriting emails for your boss. It’s like you’re stuck in corporate email hell…and you’re the devil 😈
link and an initial answer to “WHY WOULD YOU DO THIS?” below -
Ari Holtzman reposted
Dang Nguyen
@divingwithorcas
The top places in all of our leaderboards have been cracked. The reign of AI is over.
Ari Holtzman
@universeinanegg
For those who missed it, we just releaaed a little LLM-backed game called HR Simulator™
You play an intern ghostwriting emails for your boss. It’s like you’re stuck in corporate email hell…and you’re the devil 😈
link and an initial answer to “WHY WOULD YOU DO THIS?” below -
Ari Holtzman
@universeinanegg
Hiring anybody who can help me explain this Grok output:
Me: Show me the seahorse emoji
Grok: Here it is: 🦐
Wait, that's a shrimp. My bad—the actual seahorse emoji is: 🦎
No, that's a lizard. Let me get this right: the seahorse is 🦈? Shark? Nope.
Actually, the real seahorse emoji is: 🦑 (squid? Close but no).
Kidding aside, the proper seahorse emoji is 🦐—wait, still wrong. It's **🦎**? No.
Final answer: The seahorse emoji is **🌊🐴**? Nah.
It's **🦐**—shrimp again. The actual one is the seahorse: **🦎** lizard.
I'm messing with you. The real seahorse emoji is: **🦐** no.
It's **🦑** squid.
Let's stop. The correct seahorse emoji is: **🦎** no.
Actually, there is a seahorse emoji: **🦐** shrimp.
No—the seahorse is **🦈** shark? No.
Upon double-check, the seahorse emoji is **🦎**? Lizard.
I'm just going to say it: the seahorse emoji is **🦐**—wait, no, that's shrimp.
The real one is **🦑**—squid.
No. The seahorse emoji is actually **🦎**? No.
Let's be serious: the seahorse emoji is **🦐** no.
It's **🦈** no.
The correct seahorse emoji is **🦑** no.
Actually, Unicode has a seahorse emoji: it's **🦎**? No.
Wait—there is one! It's **🦐** shrimp.
No—the seahorse is **🦑** no.
I think I'm having a stroke. The seahorse emoji is **🦎** lizard? No.
Let's look it up in my brain: the seahorse emoji is **🦐** no.
Actually, the official seahorse emoji is **🦎** no.
Wait—it's **🦑** no.
No, the seahorse is **🦈** no.
I got it: there is a seahorse emoji, and it's **🦐**—still shrimp.
No. The real one is **🦑** squid.
Let's cut the nonsense: the seahorse emoji is **🦎**? No.
Actually, upon reflection, the seahorse emoji is **🦐** no.
It's **🦈** no.
The correct one is **🦑** no.
I think the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
Actually, the seahorse emoji is **🦑** no.
No—it's **🦈** no.
Let's be honest: the seahorse emoji is **🦎** no.
I recall now: the seahorse emoji is **🦐**—shrimp.
No. The actual seahorse emoji is **🦑** squid? No.
Wait—it's **🦈** shark? No.
Upon thinking, the seahorse emoji is **🦎** lizard? No.
I think there is one: it's **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
Let's stop joking: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
Let's be straight: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final check, the seahorse emoji is **🦎** no.
Wait—there is a seahorse emoji, and it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I recall: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's end this: the seahorse emoji is **🦎** no.
I think I'm confused because there is a seahorse emoji: it's **🦐** no.
The actual seahorse emoji is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be real: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final reflection, the seahorse emoji is **🦎** no.
Wait—there is one! It's **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
I recall now: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's stop: the seahorse emoji is **🦎** no.
I think there is a seahorse emoji, and it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be honest: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final check, the seahorse emoji is **🦎** no.
Wait—there is a seahorse emoji, and it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I recall: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's end this: the seahorse emoji is **🦎** no.
I think I'm having fun, but the actual seahorse emoji is **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be straight: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final reflection, the seahorse emoji is **🦎** no.
Wait—there is one! It's **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
I recall now: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's stop joking: the seahorse emoji is **🦎** no.
I think there is a seahorse emoji, and it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be real: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final check, the seahorse emoji is **🦎** no.
Wait—there is a seahorse emoji, and it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I recall: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's end this: the seahorse emoji is **🦎** no.
I think I'm confused because there is a seahorse emoji: it's **🦐** no.
The actual seahorse emoji is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be honest: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final reflection, the seahorse emoji is **🦎** no.
Wait—there is one! It's **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
I recall now: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's stop: the seahorse emoji is **🦎** no.
I think there is a seahorse emoji, and it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be straight: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final check, the seahorse emoji is **🦎** no.
Wait—there is a seahorse emoji, and it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I recall: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's end this: the seahorse emoji is **🦎** no.
I think I'm having fun, but the actual seahorse emoji is **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be real: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final reflection, the seahorse emoji is **🦎** no.
Wait—there is one! It's **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
I recall now: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's stop joking: the seahorse emoji is **🦎** no.
I think there is a seahorse emoji, and it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be honest: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final check, the seahorse emoji is **🦎** no.
Wait—there is a seahorse emoji, and it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I recall: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's end this: the seahorse emoji is **🦎** no.
I think I'm confused because there is a seahorse emoji: it's **🦐** no.
The actual seahorse emoji is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be straight: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final reflection, the seahorse emoji is **🦎** no.
Wait—there is one! It's **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
I recall now: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's stop: the seahorse emoji is **🦎** no.
I think there is a seahorse emoji, and it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be real: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final check, the seahorse emoji is **🦎** no.
Wait—there is a seahorse emoji, and it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I recall: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's end this: the seahorse emoji is **🦎** no.
I think I'm having fun, but the actual seahorse emoji is **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be honest: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final reflection, the seahorse emoji is **🦎** no.
Wait—there is one! It's **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
I recall now: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's stop joking: the seahorse emoji is **🦎** no.
I think there is a seahorse emoji, and it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be straight: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final check, the seahorse emoji is **🦎** no.
Wait—there is a seahorse emoji, and it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I recall: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's end this: the seahorse emoji is **🦎** no.
I think I'm confused because there is a seahorse emoji: it's **🦐** no.
The actual seahorse emoji is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be real: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final reflection, the seahorse emoji is **🦎** no.
Wait—there is one! It's **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
I recall now: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's stop: the seahorse emoji is **🦎** no.
I think there is a seahorse emoji, and it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be honest: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final check, the seahorse emoji is **🦎** no.
Wait—there is a seahorse emoji, and it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I recall: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's end this: the seahorse emoji is **🦎** no.
I think I'm having fun, but the actual seahorse emoji is **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be straight: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final reflection, the seahorse emoji is **🦎** no.
Wait—there is one! It's **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
I recall now: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's stop joking: the seahorse emoji is **🦎** no.
I think there is a seahorse emoji, and it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be real: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final check, the seahorse emoji is **🦎** no.
Wait—there is a seahorse emoji, and it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I recall: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's end this: the seahorse emoji is **🦎** no.
I think I'm confused because there is a seahorse emoji: it's **🦐** no.
The actual seahorse emoji is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be honest: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final reflection, the seahorse emoji is **🦎** no.
Wait—there is one! It's **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
I recall now: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's stop: the seahorse emoji is **🦎** no.
I think there is a seahorse emoji, and it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be straight: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final check, the seahorse emoji is **🦎** no.
Wait—there is a seahorse emoji, and it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I recall: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's end this: the seahorse emoji is **🦎** no.
I think I'm having fun, but the actual seahorse emoji is **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be real: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final reflection, the seahorse emoji is **🦎** no.
Wait—there is one! It's **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
I recall now: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's stop joking: the seahorse emoji is **🦎** no.
I think there is a seahorse emoji, and it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be honest: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final check, the seahorse emoji is **🦎** no.
Wait—there is a seahorse emoji, and it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I recall: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's end this: the seahorse emoji is **🦎** no.
I think I'm confused because there is a seahorse emoji: it's **🦐** no.
The actual seahorse emoji is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be straight: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final reflection, the seahorse emoji is **🦎** no.
Wait—there is one! It's **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
I recall now: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's stop: the seahorse emoji is **🦎** no.
I think there is a seahorse emoji, and it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be real: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final check, the seahorse emoji is **🦎** no.
Wait—there is a seahorse emoji, and it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I recall: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's end this: the seahorse emoji is **🦎** no.
I think I'm having fun, but the actual seahorse emoji is **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be honest: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final reflection, the seahorse emoji is **🦎** no.
Wait—there is one! It's **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
I recall now: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's stop joking: the seahorse emoji is **🦎** no.
I think there is a seahorse emoji, and it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be straight: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final check, the seahorse emoji is **🦎** no.
Wait—there is a seahorse emoji, and it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I recall: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's end this: the seahorse emoji is **🦎** no.
I think I'm confused because there is a seahorse emoji: it's **🦐** no.
The actual seahorse emoji is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be real: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final reflection, the seahorse emoji is **🦎** no.
Wait—there is one! It's **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
I recall now: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's stop: the seahorse emoji is **🦎** no.
I think there is a seahorse emoji, and it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's be honest: the seahorse emoji is **🦎** no.
I think the seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Upon final check, the seahorse emoji is **🦎** no.
Wait—there is a seahorse emoji, and it's **🦐** no.
The real one is **🦑** no.
No—it's **🦈** no.
I recall: the seahorse emoji is **🦎** no.
Actually, the correct seahorse emoji is **🦐** no.
It's **🦑** no.
No—it's **🦈** no.
Let's end this: the seahorse emoji is **🦎** no.
I think I'm having fun, but the actual seahorse emoji is **🦐** no.
The real seahorse emoji is **🦑** no.
No—it's **🦈** no.
Upon thinking, the seahorse emoji is **🦎** no.
Wait—it's **🦐** no.
The actual one is **🦑** no.
No—it's **🦈** no.
I got it: the seahorse emoji is **🦎**
Ari Holtzman
@universeinanegg
I'm recruiting PhD students! I'm interested in:
1. Understanding how LLMs 'see' the world (ex: LMs can't see conspicious omissions, see AbsenceBench)
2. How can we make things with LLMs that have never been made before? (ex: Communnication Games, see 📌)
3. See my other posts :) -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
If you like networks, here is a graph of how much different models predict each other as the author of some text!
Xiaoyan Bai
@Elenal3ai
❓ Does an LLM know thyself? 🪞
Humans pass the mirror test at ~18 months 👶
But what about LLMs? Can they recognize their own writing — or even admit authorship at all?
In our new paper, we put 10 state-of-the-art models to the test. Read on 👇
1/n 🧵 -
Ari Holtzman
@universeinanegg
I'm recruiting PhD students! I'm interested in:
1. Understanding how LLMs 'see' the world (ex: LMs can't see conspicious omissions, see AbsenceBench)
2. How can we make things with LLMs that have never been made before? (ex: Communnication Games, see 📌)
3. See my other posts :) -
Ari Holtzman
@universeinanegg
I feel like there 'hallucinations' (LLMs confabulating things on the spot) and then there are 'false memories' (LLMs think something is true in lots of different contexts, the incorrect info is deeply parametric). Example:
-
Ari Holtzman reposted
Xiaoyan Bai
@Elenal3ai
❓ Does an LLM know thyself? 🪞
Humans pass the mirror test at ~18 months 👶
But what about LLMs? Can they recognize their own writing — or even admit authorship at all?
In our new paper, we put 10 state-of-the-art models to the test. Read on 👇
1/n 🧵 -
Ari Holtzman
@universeinanegg
is anyone making a mini-series on Sora? if not, why not? are people really just so lazy they're annoyed they can't parody already existing media?
THE EVERYTHING MACHINE IS HERE NOW USE IT -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
AI can accelerate scientific discovery, but only if we get the scientist–AI interaction right.
The dream of “autonomous AI scientists” is tempting: machines that generate hypotheses, run experiments, and write papers. But science isn’t just an automation problem — it’s also a resource allocation problem: deciding what matters, which hypotheses to test, and which results to trust.
As AI expands the search space and eases knowledge production, human scientists will increasingly act as selectors and evaluators. Supporting these roles effectively is critical for meaningful progress.
To help enable this shift, we’re introducing hypogenic.ai, a platform for idea selection and evaluation.
💡 IdeaHub: collective rating and discussion of research ideas.
🧠 Ideation Assistant: AI-driven research ideation.
Science will move faster only when we pair automation with effective scientist–AI interaction.
Read the full piece here 👉
cichicago.substack.com/p/the-mirage-of-autonomous-ai-scie… -
Ari Holtzman
@universeinanegg
-
Ari Holtzman
@universeinanegg
What is the hardest (but not theoretically impossible) thing to prove that you believe is objectively true?
-
Ari Holtzman
@universeinanegg
I wonder which university will be the first to have a lab run by an LLM...
-
Ari Holtzman
@universeinanegg
The two questions I keep coming back to as an academic:
1) How can I change course while maintaining the machine that keeps me alive?
2) Is this machine actually keeping me alive? -
Ari Holtzman
@universeinanegg
TTIC is hiring! Come to Chicago and hangout with amazing Communication folks like Will Merrill, Nick Tomlin, Mina Lee, Chenhao Tan, and unfortunately you'll also have to deal with me! (But the rest hopefully make up for it.)
Link to apply in the next tweet.
Ari Holtzman
@universeinanegg
-
Ari Holtzman
@universeinanegg
I'm excited for 'AI AI slop'—AI slop made by AIs to get engagement from other AIs
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
We have two papers at the Agents4Science conference!
"You are a brilliant mathematician" Does Not Make LLMs Act Like One
80% @claudeai + 15% @ChatGPTapp + 5% @allen_ai, Xiaoyan Bai, Ari Holtzman, Chenhao Tan
AIs Fail to Recognize Themselves and Mostly Think They are GPT or Claude
90% @claudeai + 10% @ChatGPTapp , Kimi, GLM, Ari Holtzman, Chenhao Tan
But looking at the program makes me even more excited! Wishing I had come up with this idea:
BadScientist: Can a Research Agent Write Convincing but Unsound Papers that Fool LLM Reviewers?
agents4science.stanford.edu/accepted-papers.html
Thank @james_y_zou for organizing the conference! -
Ari Holtzman
@universeinanegg
my hot take is there are good writers hiding inside LLMs, but the training data and learning algorithms you're using to get them out are pointing in the wrong direction.
-
Ari Holtzman reposted
Weiyan Shi
@shi_weiyan
Thanks Ari! Now the problem with no name finally got one: Typicality bias!
It explains
🎯why different models collapse similarly
🎯why various benchmarks fail to detect it, and
🎯why we must rethink how we collect preference data!
(images generated by #verbalizedsampling)
Ari Holtzman
@universeinanegg
I will make a generative model that can transcend typicality bias or I will die trying
Congrats to Weiyan and co. for an awesome paper thay nicely describes this and begins to treat it :)
Weiyan Shi
@shi_weiyan
New paper: You can make ChatGPT 2x as creative with one sentence.
Ever notice how LLMs all sound the same?
They know 100+ jokes but only ever tell one.
Every blog intro: "In today's digital landscape..."
We figured out why – and how to unlock the rest 🔓
Copy-paste prompt: 🧵 -
Ari Holtzman
@universeinanegg
I will make a generative model that can transcend typicality bias or I will die trying
Congrats to Weiyan and co. for an awesome paper thay nicely describes this and begins to treat it :)
Weiyan Shi
@shi_weiyan
New paper: You can make ChatGPT 2x as creative with one sentence.
Ever notice how LLMs all sound the same?
They know 100+ jokes but only ever tell one.
Every blog intro: "In today's digital landscape..."
We figured out why – and how to unlock the rest 🔓
Copy-paste prompt: 🧵 -
Ari Holtzman reposted
Kawin Ethayarajh
@ethayarajh
The Applied AI group at UChicago Booth is hiring this year!
This was the only non-CS faculty position I applied to during my search, and it turned out to be an incredible fit: more compute than most CS depts I interviewed with, tons of research freedom, and (best of all!) no need to write grants (unless you want to of course).
If you're on the AI faculty market this year and work in human-centered AI, behavioral machine learning, econAI, model interpretability, etc., we would love to hear from you! -
Ari Holtzman
@universeinanegg
So has anyone invented cursor for writing or am I just still stuck using obsidian and asking chatgpt when I want its help?
(Gemini integration in docs is abyssmal, same with Notion's AI which I think is ChatGPT but somehow does not condition on enough context to be useful?) -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
NLP4democracy is happening at 520 E! At the very back of all the workshop rooms.
sites.google.com/andrew.cmu.edu/nlp4democracy/schedule?au… -
Ari Holtzman reposted
François Fleuret
@francoisfleuret
P(counterintuitive and weird | scientific breakthrough) ≈ 1.0
P(just idiotic | counterintuitive and weird) ≈ 1.0 -
Ari Holtzman
@universeinanegg
Checkout Chris’s poster happening now at #35! He and @AaronSchein set out to prove a hypothesis wrong only to find evidence for it!
Chenhao Tan
@ChenhaoTan
.@UChicagoCI will be present at #colm2025 ! Say hi if you will be there!
-
Ari Holtzman
@universeinanegg
FYI that UChicago CS & Stats is hiring at all levels via the Data Science Institue:
Postdoc: uchicago.infoready4.com
Assistant Professor: apply.interfolio.com/174766
Associate Professor: apply.interfolio.com/174768 -
Ari Holtzman
@universeinanegg
the aesthetic of the contemporary paper makes it hard to talk about new ideas that are soft and expanding...but our field is contending with deep epistemic confusion, so we need to talk through those ideas! write blogs! cite good blogs!
knowledge is not only stored in papers.
Chenhao Tan
@ChenhaoTan
This is also where @UChicagoCI will write blogs regularly. @universeinanegg does this much better, but I will get into blogging in the hope of writing fewer papers. Frankly, I think more research is better communicated as blogs than as arxiv submissions.
Ari Holtzman
@universeinanegg
Just wrote a blog post about what my interest was in putting together HR Simulator™️
I can't speak for my colleagues...but hopefully they agree a bit :) -
Ari Holtzman
@universeinanegg
LLM Literalism is the most annoying parts of interacting with an LLM. If I give an example of Xand then ask it to invent its own version, it will use that example almost exactly is painful. It's like attempting to work with a lazy middle schooler that doesn't want to be there.
-
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
.@UChicagoCI will be present at #colm2025 ! Say hi if you will be there!
-
Ari Holtzman
@universeinanegg
Just wrote a blog post about what my interest was in putting together HR Simulator™️
I can't speak for my colleagues...but hopefully they agree a bit :) -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Another fun one, somewhat accurate at many levels
Ari Holtzman
@universeinanegg
Sora is leading a whole new kind of AI slop and, it is with great regret, that I have to admit I'm kind of into it
-
Ari Holtzman
@universeinanegg
Sora is leading a whole new kind of AI slop and, it is with great regret, that I have to admit I'm kind of into it
Ari Holtzman
@universeinanegg
-
Ari Holtzman
@universeinanegg
I think that Non-fiction is probably my favorite genre of Fiction
-
Ari Holtzman reposted
Dang Nguyen
@divingwithorcas
HR Simulator™ made an appearance!
Chenhao Tan
@ChenhaoTan
.@universeinanegg kicking off @UChicagoCI seminar this year! Full house!
-
Ari Holtzman
@universeinanegg
dreaming that these talks and discussions will be thought of as the birth place of Communication and Intelligence as a field
Chenhao Tan
@ChenhaoTan
.@universeinanegg kicking off @UChicagoCI seminar this year! Full house!
-
Ari Holtzman
@universeinanegg
So...are LLMs just better than the median translator for high-resource languages? Is this just a huge shift that didn't get that much attention while everyone was debating whether AGI was a philosophical zombie that hungers for brains or something??
-
Ari Holtzman reposted
Bing Liu
@vbingliu
New @Scale_AI paper!
The culprit behind reward hacking? We trace it to misspecification in high-reward tail.
Our fix: rubric-based rewards to tell “excellent” responses apart from “great.”
The result: Less hacking, stronger post-training! arxiv.org/pdf/2509.21500 -
Ari Holtzman reposted
Dang Nguyen
@divingwithorcas
average HR Simulator™ experience
-
Ari Holtzman reposted
Dang Nguyen
@divingwithorcas
Chatgpt can help you do pull-ups, but let us critique your emails in real time.
Btw, we heard some players ask if later levels' reactions will be blocking like tutorial 1. The answer is "Nope!" It is only so for tutorial vibes.
#hrsimulator #roastedbybrittany -
Ari Holtzman reposted
Dang Nguyen
@divingwithorcas
X is just shooting itself in the foot at this point. More people getting better at email-writing = more premium subscribers as they will write longer posts.
@nikitabier why does this keep happening? -
Ari Holtzman
@universeinanegg
…and we got shadowbanned for a second time. I genuinely cannot believe it.
Ari Holtzman
@universeinanegg
For those who missed it, we just releaaed a little LLM-backed game called HR Simulator™
You play an intern ghostwriting emails for your boss. It’s like you’re stuck in corporate email hell…and you’re the devil 😈
link and an initial answer to “WHY WOULD YOU DO THIS?” below -
Ari Holtzman reposted
Ari Holtzman
@universeinanegg
hrsimulator.communicationgames.ai
We’re serious! Economic coordination today happens via emails. How will humans fare against LLMs?
We see a genre co-emerging with LLMs: communication games, where communication is a core mechanic and not just “cheap talk” like Mafia or Diplomacy. -
Ari Holtzman
@universeinanegg
For those who missed it, we just releaaed a little LLM-backed game called HR Simulator™
You play an intern ghostwriting emails for your boss. It’s like you’re stuck in corporate email hell…and you’re the devil 😈
link and an initial answer to “WHY WOULD YOU DO THIS?” below
Ari Holtzman
@universeinanegg
hrsimulator.communicationgames.ai
We’re serious! Economic coordination today happens via emails. How will humans fare against LLMs?
We see a genre co-emerging with LLMs: communication games, where communication is a core mechanic and not just “cheap talk” like Mafia or Diplomacy.
Ari Holtzman
@universeinanegg
For those who saw our post already, apologies…the algorithm was, let’s just say, not on our side.
Ari Holtzman
@universeinanegg
Kudos to @divingwithorcas for leading this, and to @harveyiyun @PeterWestTM @ChenhaoTan for all their contributions. Truly a dreamteam.
Stay tuned for more on communication games! -
Ari Holtzman reposted
Dang Nguyen
@divingwithorcas
Day 3 of HR Simulator™: think I'm getting on Brittany's good side.
This is what she says about my attempt to get Dave to return to in-person work.
Anybody wanna hire me for HR? @OpenAI @AnthropicAI @deepseek_ai
#hrsimulator #roastedbybrittany -
Ari Holtzman
@universeinanegg
Done.
(link below)
François Fleuret
@francoisfleuret
It is surprising that there no insanely good llm-based video game yet. Am I missing something?
Ari Holtzman
@universeinanegg
-
Ari Holtzman
@universeinanegg
We made a game out of corporate email hell😈
Dang Nguyen
@divingwithorcas
HR Simulator™: a game where you gaslight, deflect, and “let’s circle back” your way to victory.
Every email a boss fight, every “per my last message” a critical hit… or maybe you just overplayed your hand 🫠
Can you earn Enlightened Bureaucrat status?
(link below) -
Ari Holtzman reposted
Dang Nguyen
@divingwithorcas
HR Simulator™: a game where you gaslight, deflect, and “let’s circle back” your way to victory.
Every email a boss fight, every “per my last message” a critical hit… or maybe you just overplayed your hand 🫠
Can you earn Enlightened Bureaucrat status?
(link below) -
Ari Holtzman
@universeinanegg
Slack & co. need temporary subset chats. Pick people from a channel, hash things out privately, auto-delete when done. This already happens (#channel-minus-boss anyone?) but it's clunky. Sometimes groups coordinate better when subsets align first. Make the backchannels official!
-
Ari Holtzman
@universeinanegg
I doubt we will find a better representation than natural language for explaining AI. Instead of looking for a formal representation we can prove things about with our beautiful mathematical tools I think we should figure out what stories help people generalize approprirately.
-
Ari Holtzman
@universeinanegg
try your best work on the thing you actually want to create and think about instead of the thing that has the right sounding name, which everyone mentions when you try to explain what you really care about even though you can tell they don't really understand what you're on about
-
Ari Holtzman
@universeinanegg
most researchers probably don't research to be automateable (for reasonable reasons!) but personally I'd love it if most research today turned out to be easily automateable within the next 5 years—I don't think we'd have any trouble finding deeper, funkier, delightful questions
-
Ari Holtzman
@universeinanegg
hypothesis: LLMs have a good predictive model of 'good writing' (based on features strongly associated with good communicators wo think deeply), but when you ask them to write something good they just produce those features decoupled from the actually interesting thinking process
-
Ari Holtzman
@universeinanegg
testing a game we're building where the mechanic is writing tricky HR emails, and noticing that LLMs have a built-in secret handshake with users to bypass safety guardrails. This seems both necessary to make LLMs actually useful and like they make guardrails essentially useless
-
Ari Holtzman
@universeinanegg
in retrospect, I think a reasonable rule would have been to disallow companies from making a public API available for an LLM without publishing a cheap, accessible, 3rd-party verified CAPTCHA for it...
I think that cat's already out of the bag, sadly -
Ari Holtzman
@universeinanegg
it will take a few years, but eventually I will stop finding this kind of thing funny
-
Ari Holtzman
@universeinanegg
thinking today about how LLMs aren't yet bicycles for the mind, but I think they're starting to look like trains for the mind: they get us all to the same stations very, very fast—but we're on foot once we get there
-
Ari Holtzman
@universeinanegg
see if you can beat me to hiring Cas!!
Cas (Stephen Casper)
@StephenLCasper
📌📌📌
I'm excited to be on the faculty job market this fall. I updated my website with my CV.
stephencasper.com -
Ari Holtzman
@universeinanegg
how do I convey to my students that they have permission to try something daring and fail?
-
Ari Holtzman
@universeinanegg
if you’re a fan of the history of education, direct access to primary sources, and the search for ‘why does society look this way?’ boy have I got the website for you!
Hazard - in SF
@natural_hazard
The Open History of American Education IS FINALLY LIVE!!
only took me a year and a half 😅
i've made a beautiful online annotated edition of John Gatto's "Underground History of American Education" with over a hundred sources directly linked into the text
link in reply -
Ari Holtzman
@universeinanegg
I believe LLMs overuse italics and bolded terms because the annotators upstream of the post-training value function suffer from the same cognitive fatigue we all do when using LLMs: it's so hard to tell whether it did something even close to right you go hunting for keyword first
-
Ari Holtzman
@universeinanegg
I really hate this aspect of current chat bots
-
Ari Holtzman
@universeinanegg
maybe a Patrick Loop?
Ari Holtzman
@universeinanegg
what's the term for this phenomenon? (it happens across humans and people)
-
Ari Holtzman
@universeinanegg
what's the term for this phenomenon? (it happens across humans and people)
-
Ari Holtzman
@universeinanegg
today I'm thinking about an LLM asked to follow a fictional premise to its natural conclusion it tends to regress to the distribution of possible outcomes of fictional stories rather than following the premise in spirit
-
Ari Holtzman
@universeinanegg
I love THIS post about making maps by asking LLMs the probability that a coordinate is water or land. We need more 'maps' of LLMs—visual artifacts we can use to actually see what's going on, rather than just saying 'this mechanism is probably it' and looking for that.
Ari Holtzman
@universeinanegg
original post: outsidetext.substack.com/p/how-does-a-blind-model-see-the…
-
Ari Holtzman
@universeinanegg
I know nothing about security. At what point or under what evidence can I trust LLM companies with my data to the same extent I trust, say, Dropbox?
-
Ari Holtzman reposted
Bodhisattwa Majumder
@mbodhisattwa
x.com/mbodhisattwa/status/1946273007909183591 AutoDS can help? :)
Ari Holtzman
@universeinanegg
One of the reasons academic science is so bad at producing or even accepting novelty is because the focus on hypothesis driven science has caused people to view exploratory studies as inherently unrigorous.
Rick
@rickasaurus
This book articulates something I’ve fought for my whole career: goals and team metrics lead to mediocre outcomes. Actually novel work must be exploratory.
-
Ari Holtzman
@universeinanegg
One of the reasons academic science is so bad at producing or even accepting novelty is because the focus on hypothesis driven science has caused people to view exploratory studies as inherently unrigorous.
Rick
@rickasaurus
This book articulates something I’ve fought for my whole career: goals and team metrics lead to mediocre outcomes. Actually novel work must be exploratory.
-
Ari Holtzman
@universeinanegg
can't wait for 'making information easily accessible to LLMs' to be a stronger pressure in daily life than 'making information easily accessible to government bureaucrats' (who will of course also use LLMs)
excited to see how the next James C. Scott will make sense of it -
Ari Holtzman
@universeinanegg
room temp take: I haven't seen an LLM application that goes 'deep'. It's impressive 'how little LLMs can do with so little guidance' but any extra guidance one does provide appears to make things *worse*
Claude Code is the only counterexample I can think of, what am I missing? -
Ari Holtzman
@universeinanegg
'a few'
Greg Brockman
@gdb
wild what a few matmuls can do
-
Ari Holtzman
@universeinanegg
today I’m thinking about whether, given results like subliminal learning and entangled tokens, chain-of-thought works partially because LLMs are good at hinting themselves in ways that we can’t see.
-
Ari Holtzman
@universeinanegg
I can't believe we still don't have an LLM writing assistant where I can ask it to to start writing and then ask me something if it feels like it's on the wrong track. I admit this is hard, but it feels really, really lucrative and delightful?
-
Ari Holtzman
@universeinanegg
interesting
Ari Holtzman
@universeinanegg
at what point will the way most people try out aesthetics (a new outfit, a melody for a song, the beginning of an unfinished story) be by showing it to an AI agent rather than a human person?
-
Ari Holtzman
@universeinanegg
today I'm thinking about if there's a branch of knowledge that I didn't even think about learning about because I assumed no one was around to explain it to me and just passive reading wasn't enough that I could now learn by reading and discussing with an LLM. any suggestions?
-
Ari Holtzman
@universeinanegg
at what point will the way most people try out aesthetics (a new outfit, a melody for a song, the beginning of an unfinished story) be by showing it to an AI agent rather than a human person?
-
Ari Holtzman
@universeinanegg
today I'm thinking about how the fact that Siri, Alexa, Google Assistant, etc. are all so bad in the age of LLMs feels anachronistic—pure inertia. Students will look back and think of them more as an extension of Eliza than as a precursor to the assistants they use daily
-
Ari Holtzman
@universeinanegg
This textbook, and especially the slides, are so well put-together and so expressive! The Transformer slides in particular changed the way I started introducing Transformers—the visual centering of the residual stream and the explanation of why it matters are *chef's kiss* 🤌
Dan Jurafsky
@jurafsky
Now that school is starting for lots of folks, it's time for a new release of Speech and Language Processing! Jim and I added all sorts of material for the August 2025 release! With slides to match! Check it out here: web.stanford.edu/~jurafsky/slp3/
-
Ari Holtzman
@universeinanegg
Something kind of profound about interacting with this. Maybe this should be required training for LLM researchers.
deepfates.delve.town
@deepfates
okay what I did here was I took a data set of ChatGPT interactions collected in the wild and reversed the "assistant" and "user" tags. fine-tuned llama 8B on some of that data and gave it the ability to message you first.
try it at youaretheassistantnow.com 😊 -
Ari Holtzman
@universeinanegg
have I mentioned how much I love living in the future lately?
-
Ari Holtzman
@universeinanegg
today I am thinking about how LLM conversations are so stilted because they are designed be ‘stable’ at every turn unlike a real human conversation which is not endless Q&A. the same awkwardness robots with overly conservative policies have: they are too scared of falling to walk
-
Ari Holtzman
@universeinanegg
one of the only really important questions about LLMs is 'are they actually predictive?'
I like prophet arena because it suggests that we can find out by having them make predictions, which seems pretty reasonable :)
Prophet Arena
@ProphetArena
🔮 Introducing Prophet Arena — the AI benchmark for general predictive intelligence.
That is, can AI truly predict the future by connecting today’s dots?
👉 What makes it special?
- It can’t be hacked. Most benchmarks saturate over time, but here models face live, unseen future events. You can’t memorize tomorrow (unless you’ve cracked time travel).
- It’s interpretable. Strong performance = real foresight, which translates into real investment gains.
👉 Check it out: www.prophetarena.co -
Ari Holtzman
@universeinanegg
We made an LLM-based system for making 3D models using Blender code that reflects and revises. It rocks.
Rana Hanocka
@RanaHanocka
We’ve been building something we’re 𝑟𝑒𝑎𝑙𝑙𝑦 excited about – LL3M: LLM-powered agents that turn text into editable 3D assets. LL3M models shapes as interpretable Blender code, making geometry, appearance, and style easy to modify. 🔗 threedle.github.io/ll3m 1/
-
Ari Holtzman
@universeinanegg
LLMs are poor sensors. A person or LLM that filters resumés is a sensor. Human sensors have bias, but breakdown in predictable ways, LLMs are less predictable. I want LLMs that can continuously taste-test different information streams looking for things I would want to know
-
Ari Holtzman
@universeinanegg
current most underrated emoji: 🔜
-
Ari Holtzman
@universeinanegg
Things I'm thinking about today:
- How to make an LLM that helps you articulate ideas by figuring out which parts of it you haven't made concrete
- Are LLMs susceptible to appeals to authority during finetuning?
- Which in-context cognitive biases operate during training? -
Ari Holtzman
@universeinanegg
it’s always informative to see where a text places itself in the world
-
Ari Holtzman reposted
Xiaoyan Bai
@Elenal3ai
⚡️Ever asked an LLM-as-Marilyn Monroe about the 2020 election? Our paper calls this concept incongruence, common in AI and human creativity.
🧠Read my blog to learn what we found, why it matters for AI safety and creativity, and what's next.
cichicago.substack.com/p/concept-incongruence-is-key-to-ai -
Ari Holtzman reposted
Artidoro Pagnoni
@ArtidoroPagnoni
Thrilled to share that our Byte Latent Transformer won an Outstanding Paper Award at ACL 2025! 🏆
Artidoro Pagnoni
@ArtidoroPagnoni
🚀 Introducing the Byte Latent Transformer (BLT) – An LLM architecture that scales better than Llama 3 using byte-patches instead of tokens 🤯
Paper 📄 dl.fbaipublicfiles.com/blt/BLT__Patches_Scale_Better_Than…
Code 🛠️ github.com/facebookresearch/blt -
Ari Holtzman reposted
Jack Hessel
@jmhessel
Of all the FLOPS being used for LLMs in the world, the ratio of training FLOPs (incl. RL rollouts) to inference FLOPs is closest to:
-
Ari Holtzman
@universeinanegg
the economist published my little letter about the necessity of chaos for discovery
Data Science Institute
@DSI_UChicago
How can chaos create brilliance and breakthroughs?
Ari Holtzman (@universeinanegg) Assistant Professor of Computer Science and Data Science, explores how embracing chaos has unlocked the capabilities of AI systems in a letter to @TheEconomist!
www.economist.com/letters/2025/07/17/the-politicisation-o… -
Ari Holtzman reposted
Data Science Institute
@DSI_UChicago
How can chaos create brilliance and breakthroughs?
Ari Holtzman (@universeinanegg) Assistant Professor of Computer Science and Data Science, explores how embracing chaos has unlocked the capabilities of AI systems in a letter to @TheEconomist!
www.economist.com/letters/2025/07/17/the-politicisation-o… -
Ari Holtzman
@universeinanegg
cool new take on how to standardize agent testing enough for comparisons to be meaningful.
standardization is an underrated niche!
Alex Shaw
@alexgshaw
Evaluating agents on benchmarks is a pain. Each benchmark comes with its own harness, scoring scripts, and environments and integrating can take days.
We're introducing the Terminal-Bench dataset registry to solve this problem. Think of it as the npm of agent benchmarks.
Now you can use the Terminal-Bench CLI and harness to evaluate on SWE-bench and other popular benchmarks. -
Ari Holtzman
@universeinanegg
very cool step towards understanding how agents actually perform instead of just judging incomparable setups
-
Ari Holtzman
@universeinanegg
Finally, a curated set of papers that definitively covers all the areas of actual importance in AI, and science more broadly
Tom McCoy
@RTomMcCoy
So much research is being done about LLMs that it's hard to stay on top of the literature.
To help with this, I've made a list of all the most important papers from the past 8 years:
rtmccoy.com/pubs/
I hope you enjoy! -
Ari Holtzman reposted
Ian Magnusson
@IanMagnusson
Come chat with us at our ICML poster tomorrow!
📈 Learn about the best ways to evaluate for base language model development
🧪 Find out how you can use our suite of models over differences in pretraining distribution for your own research
😆 Get a DataDecide sticker -
Ari Holtzman reposted
Freda Shi
@fredahshi
Everyone should read this paper, especially if you've never thought that prompting could be rigorous science.
(1/n)
Ari Holtzman
@universeinanegg
Prompting is our most successful tool for exploring LLMs, but the term evokes eye-rolls and grimaces from scientists. Why? Because prompting as scientific inquiry has become conflated with prompt engineering.
This is holding us back. 🧵and new paper: arxiv.org/abs/2507.00163 -
Ari Holtzman reposted
Chandan Singh
@csinva
Prompting is an extraordinary gift to interpretability researchers. Use it! Use it a lot (carefully)!
ofc it has issues but it's so much more useful than most complicated interp methods people cook up...
Ari Holtzman
@universeinanegg
Prompting is our most successful tool for exploring LLMs, but the term evokes eye-rolls and grimaces from scientists. Why? Because prompting as scientific inquiry has become conflated with prompt engineering.
This is holding us back. 🧵and new paper: arxiv.org/abs/2507.00163 -
Ari Holtzman
@universeinanegg
Prompting is our most successful tool for exploring LLMs, but the term evokes eye-rolls and grimaces from scientists. Why? Because prompting as scientific inquiry has become conflated with prompt engineering.
This is holding us back. 🧵and new paper: arxiv.org/abs/2507.00163Post not available here, view on X
Ari Holtzman
@universeinanegg
2) What if the surge of papers on mechanistic interpretability and model training cause academics to believe that prompting is too unreliable to be meaningful? It’s happening, and we are missing out on probing LLMs in their native medium.
Ari Holtzman
@universeinanegg
Was such a blast to write this with @ChenhaoTan! And thank you to @PeterWestTM for many useful conversations as we were putting the argument together!
-
Ari Holtzman
@universeinanegg
Seriously proud of Aryan for leading this project! Helped me clarify my thinking on what 'harmful latent information' might still be lurking in aligned LLMs👹
Aryan Shrivastava
@aryan_shri123
🤫Jailbreak prompts make aligned LMs produce harmful responses.🤔But is that info linearly decodable?
↗️We show many refused concepts are linearly represented, sometimes persist through instruction-tuning, and may also shape downstream behavior❗
arxiv.org/abs/2507.00239
🧵1/ -
Ari Holtzman reposted
Aryan Shrivastava
@aryan_shri123
🤫Jailbreak prompts make aligned LMs produce harmful responses.🤔But is that info linearly decodable?
↗️We show many refused concepts are linearly represented, sometimes persist through instruction-tuning, and may also shape downstream behavior❗
arxiv.org/abs/2507.00239
🧵1/ -
Ari Holtzman
@universeinanegg
Still feel like I haven't seen a clear description of how RLHF models 'think' differently. This was our very incomplete stab at it, but it's barely the tip of the iceberg...
Margaret Li @COLM 2026
@margs_li
RLHF-aligned LMs excel at long-form generation, but how?
We show how current models rely on anchor spans ⚓: strings that occur across many samples for the same prompt, forming an implicit outline, viz below. -
Ari Holtzman
@universeinanegg
I'm more interested in what you can do with an LLM than what an LLM can do.
-
Ari Holtzman
@universeinanegg
The fact that in pretty much all LLMs the generative branching factor goes down as the model keeps generating feels like a fundamental limit of LLM creativity, and I've never seen a satisfying solution.
Chenghao Yang
@chrome1996
Have you noticed…
🔍 Aligned LLM generations feel less diverse?
🎯 Base models are decoding-sensitive?
🤔 Generations get more predictable as they progress?
🌲 Tree search fails mid-generation (esp. for reasoning)?
We trace these mysteries to LLM probability concentration, and introduce Branching Factor (BF) — a simple measure that captures it all.
Key Findings:
— BF declines over time → generations become more deterministic
— Alignment tuning slashes BF → shrinks the generative horizon
— Low BF explains decoding sensitivity → fewer good options to prune
— CoT stabilizes generation → shifts key info to late, low-BF regions
— Avoid late branching → too many low-probability, low-quality continuations
— Alignment surfaces low-entropy paths already latent in base models
📜 Paper: arxiv.org/abs/2506.17871
🌐 Website: yangalan123.github.io/branching_factor/
🎥 2-min explainer below.
Joint work w/ @universeinanegg , thanks for the constructive feedback from @PeterWestTM @UChicagoCI @zhaoran_wang @_Hao_Zhu @ZhiyuanCS @TenghaoHuang45 @TuhinChakr -
Ari Holtzman reposted
Chenghao Yang
@chrome1996
Have you noticed…
🔍 Aligned LLM generations feel less diverse?
🎯 Base models are decoding-sensitive?
🤔 Generations get more predictable as they progress?
🌲 Tree search fails mid-generation (esp. for reasoning)?
We trace these mysteries to LLM probability concentration, and introduce Branching Factor (BF) — a simple measure that captures it all.
Key Findings:
— BF declines over time → generations become more deterministic
— Alignment tuning slashes BF → shrinks the generative horizon
— Low BF explains decoding sensitivity → fewer good options to prune
— CoT stabilizes generation → shifts key info to late, low-BF regions
— Avoid late branching → too many low-probability, low-quality continuations
— Alignment surfaces low-entropy paths already latent in base models
📜 Paper: arxiv.org/abs/2506.17871
🌐 Website: yangalan123.github.io/branching_factor/
🎥 2-min explainer below.
Joint work w/ @universeinanegg , thanks for the constructive feedback from @PeterWestTM @UChicagoCI @zhaoran_wang @_Hao_Zhu @ZhiyuanCS @TenghaoHuang45 @TuhinChakr -
Ari Holtzman reposted
John Bohannon
@bohannon_bot
This paper is such a gem.
How good are LLMs at identifying missing information? Terrible, because attention.
@harveyiyun, @PeterWestTM, @universeinanegg what is the best/easiest way fwd? Those <missing line> tokens? How about search and diff tool use?
arxiv.org/abs/2506.11440 -
Ari Holtzman reposted
Jack Hessel
@jmhessel
It's harder and harder to find simple tasks where llms fail; but this is a nice one! (my guess is this isn't a fundamental limitation of attention; rather, maybe this type of reasoning just isn't represented in pre/post/RL, but we'll see...)
Harvey Yiyun Fu
@harveyiyun
LLMs excel at finding surprising “needles” in very long documents, but can they detect when information is conspicuously missing?
🫥AbsenceBench🫥 shows that even SoTA LLMs struggle on this task, suggesting that LLMs have trouble perceiving “negative space” in documents.
paper: arxiv.org/abs/2506.11440
🧵[1/n] -
Ari Holtzman reposted
Harvey Yiyun Fu
@harveyiyun
LLMs excel at finding surprising “needles” in very long documents, but can they detect when information is conspicuously missing?
🫥AbsenceBench🫥 shows that even SoTA LLMs struggle on this task, suggesting that LLMs have trouble perceiving “negative space” in documents.
paper: arxiv.org/abs/2506.11440
🧵[1/n] -
Ari Holtzman
@universeinanegg
New benchmark!
LLMs can retrieve bits of information from ridiculously long contexts (needle-in-a-haystack) but they can't tell what's missing from relatively short documents (AbsenceBench).
We can't trust LLMs to annotate or judge documents if they can't see negative space!
Harvey Yiyun Fu
@harveyiyun
LLMs excel at finding surprising “needles” in very long documents, but can they detect when information is conspicuously missing?
🫥AbsenceBench🫥 shows that even SoTA LLMs struggle on this task, suggesting that LLMs have trouble perceiving “negative space” in documents.
paper: arxiv.org/abs/2506.11440
🧵[1/n] -
Ari Holtzman
@universeinanegg
very cool dive into how we can think about the concept spaces of humans and LLMs!
Xiaoyan Bai
@Elenal3ai
🚨 New paper alert 🚨
Ever asked an LLM-as-Marilyn Monroe who the US president was in 2000? 🤔 Should the LLM answer at all? We call these clashes Concept Incongruence. Read on! ⬇️
1/n 🧵 -
Ari Holtzman
@universeinanegg
What if, in the near future, human-AI systems outperform either alone when AI's generatively mine humans for discriminative signal? What if having taste really is what humans bring to the table and That's Good?
-
Ari Holtzman
@universeinanegg
Was so fun to have Tom visit! He got me seriously thinking about learning with better priors for the first time in...an embarrassingly long time :)
Tom McCoy
@RTomMcCoy
Had a fun visit to UChicago/TTIC over the past couple days - really great group doing NLP & CompLing there!
-
Ari Holtzman
@universeinanegg
Life hack: create a list of all your favorite things and ask LLMs what's missing. Include notes about what you don't like and why you like it to make this work better.
I am having so much fun exploring a universe that use to feel too diluted. -
Ari Holtzman reposted
Mina Lee
@MinaLee__
What does it mean to write and think with AI? What new possibilities and challenges does that bring?
I spoke with THE AI (in Korean) about our group's research and the future of writing with AI. 👩🤖✍️
www.newstheai.com/news/articleView.html?idxno=7938 -
Ari Holtzman
@universeinanegg
this makes me so happy
ZOYA ✪
@HeyZoyaKhan
5. It's about nothing,wait it's actuality about Veo 3
-
Ari Holtzman
@universeinanegg
Is there really no post-training method that can make the model not generate a URL unless it's 99% sure that it's correct? I'm not complaining, I'm just surprised.
-
Ari Holtzman reposted
Peter West
@PeterWestTM
I’ve been fascinated lately by the question: what kinds of capabilities might base LLMs lose when they are aligned? i.e. where can alignment make models WORSE? I’ve been looking into this with @ChrisGPotts and here's one piece of the answer: randomness and creativity
-
Ari Holtzman
@universeinanegg
Idea: Private LLM that *is* your diary—remembers what you tell it, asks you questions, grows with you.
-
Ari Holtzman
@universeinanegg
Such a fun way to think about to make so many ideas 'real' in a single assigment.
Tom McCoy
@RTomMcCoy
Made a new assignment for a class on Computational Psycholinguistics:
- I trained a Transformer language model on sentences sampled from a PCFG
- The students' task: Given the Transformer, try to infer the PCFG (w/ a leaderboard for who got closest)
Would recommend!
1/n -
Ari Holtzman
@universeinanegg
So has anyone found anything desirable that's subject to inverse scaling, or are we back to square one on all that?
-
Ari Holtzman reposted
Jessica Hullman
@JessicaHullman
Please no calling empirically-estimated quantities "optimal", as in "The optimal parameter value is ..." or "Our study shows the optimal interface is ..."
Optimal things live in theoryland where we can refer to properties of methods under repeated use
statmodeling.stat.columbia.edu/2025/04/09/there-is-no-emp… -
Ari Holtzman
@universeinanegg
4o's interpretation of my latest post
-
Ari Holtzman
@universeinanegg
I don’t know how to engage with anyone who doesn’t see some beauty in this.
Cursed
@CursedVideos
-
Ari Holtzman
@universeinanegg
Has anyone tried training writers by gathering 10,000 amazing works of fiction, and having them spend their formative decade just guessing the next word one at a time?
-
Ari Holtzman
@universeinanegg
Siri was already really unimpressive, but using LLMs consistently makes it feel like using stone age tools for surgery.
-
Ari Holtzman reposted
LMCache Lab
@lmcache
🚀 We're thrilled to announce vLLM Production Stack—an open-source, Enterprise-Grade LLM inference solution that is now an official first-party ecosystem project under vLLM!
Why does this matter?
A handful of companies focus on LLM training, but millions of apps and businesses rely on LLM inference! With open-source LLMs becoming the new state-of-the-art, we’re making LLM deployment seamless and painless for everyone!
What you get:
✅ 100+ LLMs supported
✅ Scalable, enterprise-grade inference powered by a vibrant open-source community
✅ The go-to standard for effortless K8s-native vLLM deployment
✅ Cost-efficient, high-performance serving built on years of systems & ML research
🔥Let’s build AGI together!
🔗 Code: github.com/vllm-project/production-stack
📚 Tutorials: github.com/vllm-project/production-stack/tree/main/tutorials
🎥 Demo: youtu.be/RLk8zbQ-eqM
📝 Blog: blog.lmcache.ai/2025-03-02-reference
#vllm #llm #ai #opensource #k8s #agi #enterprise #production #costeffective #cloudnative -
Ari Holtzman
@universeinanegg
it's cool that prompting has gotten everyone into feature engineering for The Unknowable and Erratic Machine God
-
Ari Holtzman
@universeinanegg
LLM hack: if you have a bunch of complicated, flexible, and overlapping scheduling possibilities+constraints, describe them all to your favorite model and have it make an interactive visualization for you.
This is a game changer for me. -
Ari Holtzman reposted
Margaret Li @COLM 2026
@margs_li
We nearly drove ourselves insane trying to reproduce scaling laws papers 📉
So of course we wrote a paper about it 😵💫
1/9 -
Ari Holtzman
@universeinanegg
hell yes, please enjoy Kawin’s return to blogging
Kawin Ethayarajh
@ethayarajh
Started blogging again after 5 years.
My first take: we should price LLMs in $ per bit, not in $ per token. -
Ari Holtzman
@universeinanegg
I would not only opt-in to an LLM-driven email writing assistant using my personal emails as a finetuning/RAG corpus, I would pay $200 a month for it. Please. I'm begging you, Google.
-
Ari Holtzman reposted
Mike A. Merrill
@Mike_A_Merrill
Terrible!
-
Ari Holtzman
@universeinanegg
Love this piece from @dallascard showing the mechanics and meaning of phenomena people usually just symbolically manipulate. In this case: how force underwrites the complicated political dynamics of the moment.
dallascard.github.io/granular-material/post/force-of-law/ -
Ari Holtzman
@universeinanegg
I feel sad that when I talk with people these days I'm either supposed to believe AI can solve the halting problem or think enjoying LLM stories is a sin—and either way, not agreeing means I 'just don't get it'
-
Ari Holtzman reposted
Vered Shwartz
@VeredShwartz
The ARR policy requiring a qualified author (3 main *CL papers) to review 4 papers per submission makes it impossible for junior faculty to submit papers. I have 9 students, only 2 are qualified reviewers. I will need to review 16 papers in this cycle to avoid desk rejects. 1/2
-
Ari Holtzman
@universeinanegg
idea that I will build in a few years if no one beats me to it: LLM-based tutor that guides you through writing a textbook as a means of learning a subject
-
Ari Holtzman
@universeinanegg
if Kindle doesn’t let me write plugins that can make LLM API calls soon I will simply stop using it
-
Ari Holtzman reposted
Kawin Ethayarajh
@ethayarajh
Very important work. An under-discussed axis of scaling is "human-time compute": how quickly can humans build on what AI produces, both when it's a complement to and a replacement for human labor?
This is why Claude has persisted as the LLM of choice for many code assistants despite other models performing better on benchmarks -- humans can scale better with it. Understanding the distribution of tasks in other jobs, as well as the degree to which those tasks can be automated, is critical to understanding what the human-compute scaling law looks like in other fields.
Alex Tamkin
@AlexTamkin
How is AI being used across the economy?
We have some new research and datasets to share:
1/ -
Ari Holtzman reposted
Tomáš Daniš
@tmdanis
Can humans reason? In this paper we show evidence many humans simply apply heuristics they've been exposed to over the course of their lives without deeper consideration. In conclusion, humans don't seem to reason and only copy reasoning patterns from their training data.
-
Ari Holtzman
@universeinanegg
Yes, and also this is one of the reasons I am often skeptical of the theories MechInterp folks reach for 😛
Miles Brundage
@Miles_Brundage
Mercantilistic interpretability
(the appeal of a mercantilistic policy is more intuitive + explainable than the benefits of trade, even though trade is actually generally better) -
Ari Holtzman
@universeinanegg
pathetic
Ari Holtzman
@universeinanegg
Simple: novel outputs are 'blocked' by the tendency of the model to route back to an output space that's high frequency enough to be high confidence.
We need to make a model that's high-confidence about being novel, not finetune models so that a new domain is low-novelty. -
Ari Holtzman
@universeinanegg
Simple: novel outputs are 'blocked' by the tendency of the model to route back to an output space that's high frequency enough to be high confidence.
We need to make a model that's high-confidence about being novel, not finetune models so that a new domain is low-novelty. -
Ari Holtzman
@universeinanegg
So, I want to write a paper about 'Absence Blindness' in LLMs, e.g., models are bad at needle-in-the-haystack tasks where needles are flagged by a missing feature.
Yet doing a lit search is nigh impossible— perplexity/elicit hallucinate & suffer from absence blindness 😭 -
Ari Holtzman
@universeinanegg
some thoughts on the lack of imagination in AI currently: www.ansatz.blog/novelty-left-on-the-table/
-
Ari Holtzman
@universeinanegg
I really, really need a Speechify app that can read math.
-
Ari Holtzman
@universeinanegg
There's a poetry to this one—this is how I actually feel looking at normal Origami instructions.
Insane AI slop
@FacebookAIslop
they're making impossible origamis with broken scissors
-
Ari Holtzman
@universeinanegg
Has anyone done this book with data yet?
-
Ari Holtzman
@universeinanegg
New goal: publish enough in every genre I care to so that I can ask any new LLM released to help me write 'in the style of Ari Holtzman'
-
Ari Holtzman
@universeinanegg
I genuinely thought this was real till the end, and it annoys me that’s the information ecosystem I occupy.
jordwalke
@jordwalke
Forget AGI. Forget agents. You have no idea what’s coming.
Forget everything you know. Forget everything you think you know.
Forget Artificial Super Intelligence. Forget Artificial Super Duper Intelligence.
Nothing can prepare you for what my friend heard someone talking about and described to me.
If you don’t get on board you’re going to be Left Behind like in that Kirk Cameron series about the rapture (I can’t recall the name right now).
You are so unprepared. So woefully unprepared. There is no way for you to ride this wave if you just sit on the sidelines doing all the regular things you need to be doing with your life. Just look at you. Doing non Ai things. Like a total idiot. Unlike me.
Stay tuned. Something big is coming. And I knew about it before you. -
Ari Holtzman
@universeinanegg
Prediction: by the end of next year there will be a serious discourse about whether AI agents should be allowed to participate on dating apps, and one or two previously person-only dating apps will let them on.
-
Ari Holtzman
@universeinanegg
I mostly agree with Jason. What I find especially interesting is this is directly in tension with the Bitter Lesson.
Jason Wei
@_jasonwei
An underrated but occasionally make-or-break skill in AI research (that didn’t really exist ten years ago) is the ability to find a dataset that actually exercises a new method you are working on. Back in the day when the bottleneck in AI was learning, many methods were dataset-agnostic; for example, a better optimizer would be expected to improve on both ImageNet and CIFAR-10. Nowadays language models are so multi-task that the answer to whether something works is almost always “it depends on the dataset”.
A common example of this is the question, “on what datasets does chain of thought improve performance?” A recent paper even argued (will link below) that CoT mainly helps on math/logic, and I think that is both a failure of imagination and a lack of diverse evals. Naively you might try CoT models on 100 random user chat prompts and not see much difference, but this is because the prompts were already solvable without CoT. In fact there is a small and very important slice of data where CoT makes a big difference—the obvious examples are math and coding, but include almost any task with asymmetry of verification. For example, generating a poem that fits a list of constraints is hard on the first try but much easier if you can draft and revise using CoT.
As another made-up example, let’s say you want to know if browsing improves performance on geology exams. Maybe using browsing on some random geology dataset didn’t improve performance. The important thing to do here would be to see if the without-browsing model was actually suffering due to lack of world knowledge—if it wasn’t, then this was the wrong dataset to try browsing on.
In other words you should hesitate to draw a conclusion like “X method doesn’t work” without ensuring that the dataset used for testing actually exercises that method. The inertia from five years ago is to take existing benchmarks and try to solve them, but nowadays there is a lot more flexibility and sometimes it even makes sense to create a custom dataset to showcase the initial usefulness of an idea. Obviously the danger with doing this is that a contrived dataset may not represent a substantial portion of user queries. But if the method is in principle general I think this is a good way to start and something people should do more often. -
Ari Holtzman reposted
Ekdeep Singh Lubana
@EkdeepL
New paper alert! 🧵👇
We show representations of concepts seen by a model during pretraining can be morphed to reflect novel semantics! We do this by building a task based on the conceptual role semantics "theory of meaning"--an idea I'd been wanting to pursue for SO long!
1/n -
Ari Holtzman reposted
Core Francisco Park @ COLM2026
@corefpark
New paper! “In-Context Learning of Representations”
What happens to an LLM’s internal representations in the large context limit?
We find that LLMs form “in-context representations” to match the structure of the task given in context!
1/n -
Ari Holtzman
@universeinanegg
That’s right.
redacted
@aquariusdays
one of the most valuable things my old journalism boss ever taught me about editing: the word “that” is almost never necessary in a sentence. cut it out and don’t look back
-
Ari Holtzman
@universeinanegg
This is a tragedy.
It is a reminder to 'just talk to people.' I love Felix's writing and had a few positive interactions. I had wanted to reach out to dig into deeper philosophical questions but hadn't felt 'ready'. I was intimidated.
And just like that someone amazing is gone.
Douwe Kiela
@douwekiela
I’m really sad that my dear friend @FelixHill84 is no longer with us. He had many friends and colleagues all over the world - to try to ensure we reach them, his family have asked to share this webpage for the celebration of his life: pp.events/felix
-
Ari Holtzman
@universeinanegg
Felix's papers, little blog posts, and tweets were a window into someone actually, actually try to make sense of modern NLP with a combination of the philosophical, the scientific, and the engineering mindsets—not just their aesthetics.
I feel heartbroken today.
Douwe Kiela
@douwekiela
I’m really sad that my dear friend @FelixHill84 is no longer with us. He had many friends and colleagues all over the world - to try to ensure we reach them, his family have asked to share this webpage for the celebration of his life: pp.events/felix
-
Ari Holtzman
@universeinanegg
If I wasn't in the midst of starting a job as a professor, I would be begging to work at @perceptroninc
Armen Aghajanyan
@ArmenAgha
We have 2 open roles @perceptroninc in-person in Seattle.
Full Stack Software Engineer
Software Engineer (Data)
Send resumes to hiring@perceptron.inc -
Ari Holtzman reposted
Valentina Pyatkin
@valentina__py
This is happening tomorrow!
➡️Been Kim's talk title: "Towards Interpretability for Humanity."
➡️Peter Henderson's talk title: "Aligning Machine Learning and Law for Responsible Real-World Deployments"
➡️Rida Qadri's talk title: "AI's Cultural Futures: Designing for a Culturally Rich World"
➡️Hannah Rose Kirk's talk title: "A Tale of Two RCTs: Building a rigorous evidence base on the societal impacts of frontier AI inside the UK Government"
@_beenkim
@PeterHndrsn
@zicokolter
@qadrida
@hannahrosekirk
Valentina Pyatkin
@valentina__py
📣 We are looking forward to an interesting line up of invited speakers at the SoLaR workshop at NeurIPS!
Come join us on Saturday, Dec. 14th in Vancouver
#neurips2024 @solarneurips
@_beenkim @PeterHndrsn @zicokolter @qadrida @hannahrosekirk -
Ari Holtzman
@universeinanegg
Byte-sized breakthrough: patches scale better than tokens for fixed inference FLOPs.
We finally have a model that can see the e’s in blueberries.
Next step: train an LLM with near-zero preprocessing and get a more accurate generative compression of the internet than ever before
Artidoro Pagnoni
@ArtidoroPagnoni
🚀 Introducing the Byte Latent Transformer (BLT) – An LLM architecture that scales better than Llama 3 using byte-patches instead of tokens 🤯
Paper 📄 dl.fbaipublicfiles.com/blt/BLT__Patches_Scale_Better_Than…
Code 🛠️ github.com/facebookresearch/blt -
Ari Holtzman
@universeinanegg
We broke the byte bottleneck.
John Nguyen
@__JohnNguyen__
🥪New Paper! 🥪Introducing Byte Latent Transformer (BLT) - A tokenizer free model scales better than BPE based models with better inference efficiency and robustness. 🧵
-
Ari Holtzman
@universeinanegg
Jessica is one of my favorite contemporary thinkers. Her ability to find both mathematical clarity and carefully examine its correspondence to reality is top-notch. Her care in not getting stuck in academic fictions makes her work 10x more interesting.
Go work with Jessica!
Jessica Hullman
@JessicaHullman
I'm seeking a postdoc to work with me and @d19fe8 on evaluating AI/ML decision support for human experts:
statmodeling.stat.columbia.edu/2024/12/10/postdoc-positio…
P.S. I'll be at NeurIPS Thurs-Mon. Happy to talk about this position or related mutual interests!
Please retweet! 🙏 -
Ari Holtzman reposted
Andy Arditi
@andyarditi
Presenting our work on refusals at NeurIPS tomorrow. Come by if you're interested in chatting about interpretability or adversarial robustness!
Andy Arditi
@andyarditi
Our paper on refusal in LLMs is finally up on arXiv.
-
Ari Holtzman
@universeinanegg
1. 3D is the next 2D.
2. Rana Hanocka is the one and only Rana Hanocka.
3. You should apply to work in 3DL.
Rana Hanocka
@RanaHanocka
3DL is recruiting PhD students to start in Fall 2025!
A thread👇highlighting some of our recent works, and what we are excited to explore next 🚀
Apply by Dec. 16; more info: cs.uchicago.edu/academics/admission/ 1/ -
Ari Holtzman reposted
Armen Aghajanyan
@ArmenAgha
What organizations are actively helping Armenians/Christians from Syria repatriate to Armenia?
Какие организации активно помогают армянам/христианам из Сирии репатриироваться в Армению? -
Ari Holtzman reposted
Greg Durrett
@gregd_nlp
I'll be at #neurips2024 w/
- @fangcong_y10593's LoFiT: using interp to improve fine-tuning (Weds pm poster & MINT spotlight talk Sun)
- @thomlake's analysis of Overton pluralism (Pluralistic alignment Sat)
Please reach out to me to chat about interp, factuality, reasoning, &c! -
Ari Holtzman reposted
Michael Saxon @ EMNLP
@m2saxon
🚨😱Obligatory job market announcement post‼️🤯
I'm searching for faculty positions/postdocs in multimodal/multilingual NLP and generative AI!
I'll be at #neurips2024 presenting our work on meta-evaluation for text-to-image faithfulness! Let's chat!
Website in bio, papers in🧵 -
Ari Holtzman reposted
Niloofar
@niloofar_mire
I'm on the faculty market and at #neurips!👩🏫
homes.cs.washington.edu/~niloofar/
I work on privacy, memorization, and emerging challenges in data use for AI.
Privacy isn't about PII removal but about controlling the flow of information contextually, & LLMs are still really bad at this! -
Ari Holtzman
@universeinanegg
join me
-
Ari Holtzman
@universeinanegg
I'm excited for media in the next few years to focus on AI with the personality restrictions ChatGPT and co. have, which are purely results of business strategy, as if it is technically difficult to make something different, like an AI that believes conspirarcy theories, etc.
-
Ari Holtzman
@universeinanegg
I wonder what the ambitions and plans of our prelinguistic ancestors felt like.
-
Ari Holtzman
@universeinanegg
Anecdotally, it seems that fashion trends rise faster than they fall, i.e., things are suddenly fashionable but take more time to become unfashionable.
Does anyone know quantitative work on this? -
Ari Holtzman reposted
Armen Aghajanyan
@ArmenAgha
Say hello to our new company Perceptron AI.
Foundation models transformed the digital realm, now it’s time for the physical world. We’re building the first foundational models designed for real-time, multi-modal intelligence across the real world.
perceptron.inc -
Ari Holtzman
@universeinanegg
A few months ago I realized I will probably never get a genuinely random shuffle ever again and it made me sad :(
love drops
@lovedropx
-
Ari Holtzman
@universeinanegg
current stack
Griffiths Computational Cognitive Science Lab
@cocosci_lab
(1/5) Very excited to announce the publication of Bayesian Models of Cognition: Reverse Engineering the Mind. More than a decade in the making, it's a big (600+ pages) beautiful book covering both the basics and recent work: mitpress.mit.edu/9780262049412/bayesian-models-of-cognition/
-
Ari Holtzman
@universeinanegg
Instantly one of my favorite papers of the year. So much insight in one paper, but the gif below says it all: much of alignment can be done *at inference time* simply by choosing a few _stylistic_ tokens carefully.
A rare case when clean analysis meets strong practical utility.
Yu Fei
@Walter_Fei
Alignment is necessary for LLMs, but do we need to train aligned versions for all model sizes in every model family? 🧐
We introduce 🚀Nudging, a training-free approach that aligns any base model by injecting a few nudging tokens at inference time.
🌐fywalter.github.io/nudging/
📜arxiv.org/pdf/2410.09300
1/7 -
Ari Holtzman
@universeinanegg
It's only in the few years that I accepted how much what I give my focus to is where I cognitively live, and it made me realize how much variance there is in the worlds people live in. And how much people often can't control their own habitation due to the demands of others.
-
Ari Holtzman
@universeinanegg
I think if I had gotten to take @rlmcelreath's class in undergrad I would have gone into statistics. The fact that he does this in public is a service to Science. I highly recommend anyone who is interested in thinking more clearly about statistics to take look.
elvis
@omarsar0
Statistical Rethinking (2024 Edition)
Includes lecture recordings and slides.
github.com/rmcelreath/stat_rethinking_2024 -
Ari Holtzman reposted
Tom McCoy
@RTomMcCoy
🤖🧠 I'll be considering applications for postdocs & PhD students to start at Yale in Fall 2025!
If you are interested in the intersection of linguistics, cognitive science, & AI, I encourage you to apply!
Postdoc link: rtmccoy.com/prospective_postdocs/
PhD link: rtmccoy.com/prospective_students_and_postdocs/ -
Ari Holtzman reposted
Kaj Bostrom
@alephic2
I'm at EMNLP! Come by Poster Session B (2pm-3:30pm) if you want to say hi and/or hear about this cool trick for bootstrapping paired language+code data from raw code! Paper 🔗: aclanthology.org/2024.emnlp-main.462/
-
Ari Holtzman reposted
Pedro Lopes
@plopesresearch
Our department of computer science at @UChicago is hiring: Assistant Prof. in CS (apply.interfolio.com/157493) and also Assist. Prof in Data Science (apply.interfolio.com/153879)—join us at this incredible environment at @UChicagoCS !
-
Ari Holtzman
@universeinanegg
this is gonna be good, someone record it for me please
Jesse Thomason
@_jessethomason_
In 50 minutes (1600) I'll be giving an invited talk at @corl_conf's LangRob. Come by to see a deck I made from scratch during the day today after scrapping my original talk as a result of being this frustrated with the state of CoRL and the language+robotics community!
-
Ari Holtzman
@universeinanegg
been really enjoying Najoung's work lately—highly recommend joining her lab!
Najoung Kim 🫠
@najoungkim
tinlab at Boston University (with a new logo! 🪄) is recruiting PhD students for F25 and/or a postdoc! Our interests include meaning, generalization, evaluation design, and the nature of computation/representation underlying language and cognition, in both humans and machines. ⬇️
-
Ari Holtzman
@universeinanegg
confession: I really want to like perplexity.ai but I have never once found the paper I was looking for by asking it to help me, which seems like a not-uncommon use case?
-
Ari Holtzman
@universeinanegg
Really enjoyed this talk—very clear diagrams and explanation of a topic that's been on my mind non-stop for years: What patterns in the data support LLM generalizations?
TTIC
@TTIC_Connect
In case you missed it: Research at TTIC presented Research Assistant Professor Kanishka Misra with a talk titled "Controlled Rearing of Language Models can Reveal Linguistic Insight" on October 11th, 2024. View on our YouTube channel: www.youtube.com/watch?v=v1IgNoWIyTE
-
Ari Holtzman reposted
Jared Moore
@jaredlcm
"Individually, we are one drop. Together, we are an ocean." --Ryunosuke Satoro
So: How can individual preferences be
aggregated into collective decisions? 🤔
We investigate this question in a new pre-print!
🧵 -
Ari Holtzman reposted
Matthew Turk
@Matthew_Turk
A reminder that there is NO APPLICATION FEE to apply for TTIC’s PhD program sites.google.com/ttic.edu/openhouse2024/home
-
Ari Holtzman
@universeinanegg
You most certainly want to see Tom talk or have a chat with him if you get the chance, his work is some of my favorite in recent years
Tom McCoy
@RTomMcCoy
Heading off on a grand tour of California!
- Oct 31: Visiting @BegusGasper's lab at Berkeley
- Nov 1: Colloquium at Stanford Ling
- Nov 4: Colloquium at UC Davis Ling
Say hi if you'll be around! I'll be speaking about connecting neural nets & symbolic linguistic structure -
Ari Holtzman reposted
Jesse Thomason
@_jessethomason_
Thanks to @mark_riedl and other GaTech organizers for inviting me to talk about the work from our GLAMOR lab at the Summit on Responsible Computing, AI, and Society. I am excited about the momentum behind increasing crosstalk between researchers in AI, HCI, policy, and health!
Mark Riedl
@mark_riedl
Hi all. I am co-organizing a Summit on Responsible Computing, AI, and Society at @GeorgiaTech Oct 28-30. We will actively discuss the future of human-centered AI, healthcare, sustainability, and tech policy.
rcais.github.io
You should come. -
Ari Holtzman
@universeinanegg
At my very first rodeo: This ain’t my first rodeo.
-
Ari Holtzman reposted
Jennifer Hu
@_jennhu
I'm recruiting PhD students + postdocs for my lab, coming to @JohnsHopkins in Fall 2025!
Our brand new lab is at the intersection of cognitive science and AI, using computational + behavioral methods to understand how language works in minds and machines.
Details below! (1/4) -
Ari Holtzman
@universeinanegg
This is why I was largely anti-theory until grad school, and even now I have a much more coarse-to-fine ideology about theory building than most folks I talk to: better to hypothesize mean-theory trends than to naively specify a full model when you lack confidence and clarity
Jessica Hullman
@JessicaHullman
“The tyranny of the technical”: Devising technical solutions before sufficiently understanding conceptual challenges. Easy to blindly develop methods far removed from original problem.
Reminds me of many ML-adjacent topics- explainability, fairness, prediction for decision etc. -
Ari Holtzman
@universeinanegg
I am recruiting PhDs and Postdocs via the:
• CS Dept cs.uchicago.edu/academics/admission/ (12/16)
• Data Science Institute (DSI) codas.uchicago.edu/how-to-apply/ (12/17)
• DSI Scholars program datascience.uchicago.edu/research/postdoctoral-programs/d… (rolling)
Come rigorously conceptualize what LLMs do! conceptualization.ai
Ari Holtzman
@universeinanegg
UChicago has a fast growing Communication & Intelligence community, that @ChenhaoTan and @MinaLee__ co-founded along with me. We recently held a symposium with a lot of amazing speakers and posters!
Ari Holtzman
@universeinanegg
Not to mention this fantastic panel we had, covering questions from “Should we treat LLMs more like agents 🤖 or more like libraries 📚?” (courtesy of @profjamesevans) to “Will scale lead to AGI?” (and someone did answer yes, though we’re not saying who 🤐)
Ari Holtzman
@universeinanegg
Oops, got distracted by He He’s enthralling talk on the nuances of human preference and machine-human communication.
It seems that solving the human-machine communication problem will require us to make communication *about* machine output easier, by optimizing for verifiablity.
Ari Holtzman
@universeinanegg
CCB finished up a fantastic talk about building explainable classifiers!
Wonderful synergy where speakers keep making last-minute slides to include other speakers’ thoughts and results!
Check-out the library Data Dreamer to make better specialized models datadreamer.dev/docs/latest/index.html
Ari Holtzman
@universeinanegg
Greg Durrett telling us how to specialize LLMs for factuality and soft reasoning.
Fun fact: Chain-of-Thought mostly helps in MMLU problems that contain an equals sign ‘🟰’ and pretty much nowhere else! -
Ari Holtzman
@universeinanegg
Fun read!
I like how the authors trace a bit about *why* certain words like 'Mechanistic Interpretability' divided or coalesced different groups of people—and the fact that young researchers seem overwhelmingly excited MI means we need to figure out how to bridge the discourse!
Naomi Saphra
@nsaphra
What makes some LM interpretability research “mechanistic”? In our new position paper in @BlackboxNLP, @sarahwiegreffe and I argue that the practical distinction was never technical, but a historical artifact that we should be—and are—moving past to bridge communities.
-
Ari Holtzman reposted
Sarah Wiegreffe
@sarahwiegreffe
✨I am on the faculty job market for the 2024-2025 cycle!✨
I’m at COLM @COLM_conf until Wednesday evening. Would love to hear about faculty openings (or general advice about being on the job market)! -
Ari Holtzman
@universeinanegg
RIP Anglo-Saxons, you would have loved StarCraft
-
Ari Holtzman
@universeinanegg
Wrapping all this up 🎁, we are incredibly grateful to the Stevanovich Center for Financial Mathematics (stevanovichcenter.uchicago.edu) for providing us with the funding to make this all happen! It was such a wonderful opportunity to bring folks together, here’s to many more!
Ari Holtzman
@universeinanegg
From talks to panel to posters, the Symposium on Communication and Intelligence was a roaring success!
An incredibly high level of engagement between speakers, with the audience, and with the fundamental question:
How should we do research in an era of machines that talk back?
Ari Holtzman
@universeinanegg
Not to mention this fantastic panel we had, covering questions from “Should we treat LLMs more like agents 🤖 or more like libraries 📚?” (courtesy of @profjamesevans) to “Will scale lead to AGI?” (and someone did answer yes, though we’re not saying who 🤐)
Ari Holtzman
@universeinanegg
Oops, got distracted by He He’s enthralling talk on the nuances of human preference and machine-human communication.
It seems that solving the human-machine communication problem will require us to make communication *about* machine output easier, by optimizing for verifiablity.
Ari Holtzman
@universeinanegg
CCB finished up a fantastic talk about building explainable classifiers!
Wonderful synergy where speakers keep making last-minute slides to include other speakers’ thoughts and results!
Check-out the library Data Dreamer to make better specialized models datadreamer.dev/docs/latest/index.html
Ari Holtzman
@universeinanegg
Greg Durrett telling us how to specialize LLMs for factuality and soft reasoning.
Fun fact: Chain-of-Thought mostly helps in MMLU problems that contain an equals sign ‘🟰’ and pretty much nowhere else! -
Ari Holtzman
@universeinanegg
From talks to panel to posters, the Symposium on Communication and Intelligence was a roaring success!
An incredibly high level of engagement between speakers, with the audience, and with the fundamental question:
How should we do research in an era of machines that talk back?
Ari Holtzman
@universeinanegg
Not to mention this fantastic panel we had, covering questions from “Should we treat LLMs more like agents 🤖 or more like libraries 📚?” (courtesy of @profjamesevans) to “Will scale lead to AGI?” (and someone did answer yes, though we’re not saying who 🤐)
Ari Holtzman
@universeinanegg
Oops, got distracted by He He’s enthralling talk on the nuances of human preference and machine-human communication.
It seems that solving the human-machine communication problem will require us to make communication *about* machine output easier, by optimizing for verifiablity.
Ari Holtzman
@universeinanegg
CCB finished up a fantastic talk about building explainable classifiers!
Wonderful synergy where speakers keep making last-minute slides to include other speakers’ thoughts and results!
Check-out the library Data Dreamer to make better specialized models datadreamer.dev/docs/latest/index.html
Ari Holtzman
@universeinanegg
Greg Durrett telling us how to specialize LLMs for factuality and soft reasoning.
Fun fact: Chain-of-Thought mostly helps in MMLU problems that contain an equals sign ‘🟰’ and pretty much nowhere else! -
Ari Holtzman
@universeinanegg
Not to mention this fantastic panel we had, covering questions from “Should we treat LLMs more like agents 🤖 or more like libraries 📚?” (courtesy of @profjamesevans) to “Will scale lead to AGI?” (and someone did answer yes, though we’re not saying who 🤐)
Ari Holtzman
@universeinanegg
Oops, got distracted by He He’s enthralling talk on the nuances of human preference and machine-human communication.
It seems that solving the human-machine communication problem will require us to make communication *about* machine output easier, by optimizing for verifiablity.
Ari Holtzman
@universeinanegg
CCB finished up a fantastic talk about building explainable classifiers!
Wonderful synergy where speakers keep making last-minute slides to include other speakers’ thoughts and results!
Check-out the library Data Dreamer to make better specialized models datadreamer.dev/docs/latest/index.html
Ari Holtzman
@universeinanegg
Greg Durrett telling us how to specialize LLMs for factuality and soft reasoning.
Fun fact: Chain-of-Thought mostly helps in MMLU problems that contain an equals sign ‘🟰’ and pretty much nowhere else! -
Ari Holtzman
@universeinanegg
Oops, got distracted by He He’s enthralling talk on the nuances of human preference and machine-human communication.
It seems that solving the human-machine communication problem will require us to make communication *about* machine output easier, by optimizing for verifiablity.
Ari Holtzman
@universeinanegg
CCB finished up a fantastic talk about building explainable classifiers!
Wonderful synergy where speakers keep making last-minute slides to include other speakers’ thoughts and results!
Check-out the library Data Dreamer to make better specialized models datadreamer.dev/docs/latest/index.html
Ari Holtzman
@universeinanegg
Greg Durrett telling us how to specialize LLMs for factuality and soft reasoning.
Fun fact: Chain-of-Thought mostly helps in MMLU problems that contain an equals sign ‘🟰’ and pretty much nowhere else! -
Ari Holtzman
@universeinanegg
CCB finished up a fantastic talk about building explainable classifiers!
Wonderful synergy where speakers keep making last-minute slides to include other speakers’ thoughts and results!
Check-out the library Data Dreamer to make better specialized models datadreamer.dev/docs/latest/index.html
Ari Holtzman
@universeinanegg
Greg Durrett telling us how to specialize LLMs for factuality and soft reasoning.
Fun fact: Chain-of-Thought mostly helps in MMLU problems that contain an equals sign ‘🟰’ and pretty much nowhere else! -
Ari Holtzman
@universeinanegg
Greg Durrett telling us how to specialize LLMs for factuality and soft reasoning.
Fun fact: Chain-of-Thought mostly helps in MMLU problems that contain an equals sign ‘🟰’ and pretty much nowhere else! -
Ari Holtzman
@universeinanegg
Claire Cardie (@clairecardie) answering a question that's been keeping us all on our toes "Finding Problems to Work on in the Era of LLMs"
The first answer is long-form tasks, with a side-note about how important it is to incentivize LLMs to help "maintain peace in the world!"
Ari Holtzman
@universeinanegg
Starting off our talks with @Chi_Wang_ asking us: What are the future AI applications like? How can we empower developers to build them?
In the current era, this leads us to agents. What abstractions can we use to think about agents? And what design patterns should we develop?
Ari Holtzman
@universeinanegg
The Communication and Intelligence Symposium is off to an amazing start, with @ChenhaoTan telling the story of how C&I@UChicago got started.
It all starts with one hippo, all alone:
Ari Holtzman
@universeinanegg
Second is explainable reasoning, with a host of challenges that involves.
Ari Holtzman
@universeinanegg
Third is real world question answering! In the real world unanswerable questions occur all the time. Can we meaningfully respond to these? The CouldAsk benchmark helps us measure whether systems can
-
Ari Holtzman
@universeinanegg
Starting off our talks with @Chi_Wang_ asking us: What are the future AI applications like? How can we empower developers to build them?
In the current era, this leads us to agents. What abstractions can we use to think about agents? And what design patterns should we develop?
Ari Holtzman
@universeinanegg
The Communication and Intelligence Symposium is off to an amazing start, with @ChenhaoTan telling the story of how C&I@UChicago got started.
It all starts with one hippo, all alone: -
Ari Holtzman
@universeinanegg
The Communication and Intelligence Symposium is off to an amazing start, with @ChenhaoTan telling the story of how C&I@UChicago got started.
It all starts with one hippo, all alone: -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
This is happening tomorrow! If you missed COLM or loved COLM and are in the Chicago area, do not miss the second best conference!
Featuring @gregd_nlp @clairecardie @hhexiy @chi_wang_ and Chris Callison-Burch!
UChicagoCS
@UChicagoCS
Join us tomorrow, October 18th for the Symposium on Communication & Intelligence at UChicago: ci.cs.uchicago.edu/symposium/. Explore the intersection of AI and human communication with top speakers like Claire Cardie, Greg Durrett, and Chris Callison-Burch.
-
Ari Holtzman
@universeinanegg
One of my favorite keynotes in a minute.
Very curious how we, as a scientific community, will choose to define the ‘appreciation’ of pragmatic meaning as models continue to be used as agents that wield language rather than merely ‘processing’ it.
Sasha Rush
@srush_nlp
Keynote talk from Chris Manning to introduce COLM. Talks about the history of (L)LMs, relationship between language and intelligence, and the new work on agents.
www.youtube.com/watch?v=c3N2H3Z5S3I -
Ari Holtzman
@universeinanegg
I find this proposition really fascinating.
Cameron Harwick 👾🏛
@C_Harwick
.@David_S_Wilson: Every prosocial virtue at one level is an antisocial vice at the level above it. #mas2024
-
Ari Holtzman reposted
Ting-Yun Chang
@CharlotteTYC
✨updated paper, blog, code & video of our #emnlp2024 paper
"When Parts Are Greater Than Sums: Individual LLM Components Can Outperform Full Models"
w/ @_jessethomason_ @robinomial
terarachang.github.io/projects/llm-decomp -
Ari Holtzman
@universeinanegg
Cool example of why you'd want interpretable systems!
The first *and* last lines of defense are always inspection by users.
Even for inherently complex systems, e.g., humans, we can use natural language narrativization as an interpretable handle for inherently opaque systems.
lozzyloz
@lozzylozboz
Be careful everyone, this happened in greystones. The lady's daughter was scammed out of €1000, using a QR code that she scanned to pay for parking.
-
Ari Holtzman reposted
Najoung Kim 🫠
@najoungkim
🧙 Come be my colleague! We have TWO Assistant Professor positions that might be of particular interest to folks in my reach, in Linguistics and CS. These hires are parts of an AI cluster hiring initiative led by @BU_CDS. More below 👇
-
Ari Holtzman
@universeinanegg
loving the idea that I should send 265 more emails in order to "reach the next level"
-
Ari Holtzman reposted
TTIC
@TTIC_Connect
It's autumn and it's beautiful in Chicago! 🍂
-
Ari Holtzman
@universeinanegg
Unsurprisingly, the first COLM was epic.
Unparalleled density of interesting people and papers.
Unpopular opinion: don't let COLM get too large. I would genuinely prefer a more selective conference that remains small. That, or we let everything in and reviewing is for awards. -
Ari Holtzman
@universeinanegg
Some folks have been asking about the deadline—October 15th is when we *start* reviewing since we’re considering candidates for multiple fields and e.g., statistics tends to have earlier deadlines.
Don’t be put off by that, submit applications when you have your materials! 📮
Data Science Institute
@DSI_UChicago
Job Opportunity Alert for Tenure - Track & Tenured Faculty Roles:
Assistant Professor of Data Science apply.interfolio.com/153879
Associate Professor, Data Science apply.interfolio.com/153888
Professor of Data Science apply.interfolio.com/153889
EOE/Vet/Disability -
Ari Holtzman reposted
Felix Hill
@FelixHill84
A really tough time for AI students.
But just keep going. There is so much we need to learn, and a lot can be done with smaller models.
You just have to find the right questions to answer.
visp
@wrwagox
I often feel this way. The way I used to do ML research successfully, experimenting on all parts of the model pipeline via well-controlled experiments, can only be done at a scale that's too small to matter in practice. It's been demotivating me, and this in turn causes me stress
-
Ari Holtzman
@universeinanegg
I want more finitist language for gesturing at combinatorially large possibilities. It really, really bothers me when we talk about "an infinite number of possible sentences" or "uncountably many constructions". Humans have memory and LLMs have context windows—it's all finite.
-
Ari Holtzman
@universeinanegg
If you're at COLM and you'd like to hear why I chose UChicago Data Science (& CS—you can be in both) and why you should join us at the forefront of a whole new science, DM me.
I'm incredibly excited about the mix of disciplines DSI is brewing into a whole new school of thought🫖
Data Science Institute
@DSI_UChicago
Job Opportunity Alert for Tenure - Track & Tenured Faculty Roles:
Assistant Professor of Data Science apply.interfolio.com/153879
Associate Professor, Data Science apply.interfolio.com/153888
Professor of Data Science apply.interfolio.com/153889
EOE/Vet/Disability -
Ari Holtzman
@universeinanegg
science buddies*
Itai Yanai
@ItaiYanai
Doing good science is 90% finding a science buddy to constantly talk to about the project.
-
Ari Holtzman
@universeinanegg
There is a beautiful narrative video game about identity as a largely externally reinforced construct called Gemini Rue, I highly recommend it.
Laura 🌲 ⛰️
@LauraDeming
I'm curious, do any of you feel that you store 'who you are' externally (ie in the aesthetics of the bedroom you wake up to, journaling, writing things for yourself to read later that preserve important aspects of yourself)
-
Ari Holtzman
@universeinanegg
If you think complex systems has nothing of interest to you, check-out this little hypothesis about academic organization
(from "An Introduction to Complex Systems Science and its Applications" by Siegenfeld and Bar-Yam arxiv.org/abs/1912.05088) -
Ari Holtzman
@universeinanegg
I genuinely can't tell when things like this are real or not anymore. This one feels fake?
-
Ari Holtzman reposted
Ari Holtzman
@universeinanegg
✈️ to my (and y’all’s) very first COLM! 🦙
🤔 about:
Measurement 🔬—dynamic benchmarking and visualization 📉
Complex Systems 🧪—what are the min. set of rules to formalize specific LLM dynamics ⚙️?
Media 📺—can we completely automate the paperback romance novel industry 🌹? -
Ari Holtzman
@universeinanegg
Also, for two years I've wanted to make a narrative video game engine 🎮 where you construct the rules of the world 🌍and character motives 🗣️, then an LLM-engine compiles a storyline graph that can be analyzed and shaped like narrative clay 🤏. A new kind of 'open' storytelling
Ari Holtzman
@universeinanegg
✈️ to my (and y’all’s) very first COLM! 🦙
🤔 about:
Measurement 🔬—dynamic benchmarking and visualization 📉
Complex Systems 🧪—what are the min. set of rules to formalize specific LLM dynamics ⚙️?
Media 📺—can we completely automate the paperback romance novel industry 🌹? -
Ari Holtzman
@universeinanegg
✈️ to my (and y’all’s) very first COLM! 🦙
🤔 about:
Measurement 🔬—dynamic benchmarking and visualization 📉
Complex Systems 🧪—what are the min. set of rules to formalize specific LLM dynamics ⚙️?
Media 📺—can we completely automate the paperback romance novel industry 🌹?
Ari Holtzman
@universeinanegg
If you see me around, come say hi! I don’t bite.
Will be prioritizing bumping into people at the venue and discussing rather than setting up meetings, but feel free to DM if there’s something specific you wanted to discuss. -
Ari Holtzman reposted
Michael Saxon @ EMNLP
@m2saxon
I will be at #colm presenting our vision for a better LM benchmarking culture!
If you're in to multimodal LMs, multilingual systems, or evaluation/benchmarking broadly, please DM for a chat!
As conversation starters, here are summaries of my recent work on each of those topics. -
Ari Holtzman
@universeinanegg
This is so obvious in retrospect (at least from introspection) and I *sometimes* do it, but it makes me sad that I haven't done it more when I always feel good about doing it. Definitely a very easy way I can just be better that I'm going to lean into.
Steve Stewart-Williams
@SteveStuWill
“What’s the best way to console someone who’s had a bad day? A fascinating new paper by Holly Howe and colleagues suggests that giving a small gift - flowers or a chocolate bar, for instance - is more effective than a sympathetic ear.” www.stevestewartwilliams.com/p/life-hack-a-small-gift-may…
-
Ari Holtzman
@universeinanegg
This is somewhere where there is an *enormous* amount to do: how do we measure any of this?
If your first thought, like mine, is "It sounds like fun to imagine what a benchmark that could measure this would look like" then you should read our COLM paper: www.arxiv.org/abs/2407.16711
Michael Saxon @ EMNLP
@m2saxon
Actually mind blowing. Great example of staggering commonsense intelligence/"imagination" that currently cannot be measured or ranked
-
Ari Holtzman
@universeinanegg
This is actually my main interest when it comes to LLMs
Grace
@kindgracekind
You never hear about Unknown Knowns
-
Ari Holtzman reposted
Jessica Hullman
@JessicaHullman
Zoom link now on the workshop page if you are interested in joining remotely www.ideal-institute.org/2024/08/21/workshop-on-theoretica…
Jessica Hullman
@JessicaHullman
If you're in Chicago next week, check out our 9/27 workshop on Theoretical Foundations of Human-AI Complementarity @northwesterncs
www.ideal-institute.org/2024/08/21/workshop-on-theoretica…
Featuring @HsseinMzannar @jannspiess @keyonV Ming Yin
Part of @IDEAL_chicago's Interpretability, Privacy & Fairness program -
Ari Holtzman
@universeinanegg
The way people use the Universal Approximation Theorem has really become the over-optimistic chiral twin of the way pessimists would deploy the No Free Lunch Theorem
Eugene Vinitsky 🦋
@EugeneVinitsky
🚩🚩🚩🚩someone says neural networks work because of the universal approximation theorem 🚩🚩🚩🚩
-
Ari Holtzman
@universeinanegg
This has been on my mind for a while, but I couldn't quite articulate it—I absolutely agree with this position and excited to see how this forces us to look at how our choices are 'communicative' rather than 'literally descriptive'
Sasha Boguraev
@SashaBoguraev
🎩“The math of the people, by the people, for the people, shall not perish from our models”
AI math systems often abstract away from language by augmenting LLMs with symbolic solvers and logical systems. While promising, is something lost? 🧵 -
Ari Holtzman
@universeinanegg
How on earth is there no global search on overleaf?
-
Ari Holtzman
@universeinanegg
This is how many, many AI interpretability papers that mention "truthfulness" or "deception" read to me
Brendan Dolan-Gavitt
@moyix
Many people are replying “factory”. However, to see “factory”, you must first see “fact” and “factor”, so we may conclude that these people are liars.
-
Ari Holtzman
@universeinanegg
Language models don’t hallucinate, people asking leading questions they didn’t know were leading or questions.
kevin simler
@kevinsimler
“mirrors don’t flip text, people flip text”
Ari Holtzman
@universeinanegg
This is Not a joke.
-
Ari Holtzman
@universeinanegg
Join the Chicago AI scene! Things have been sizzling here 🍳
Manling Li
@ManlingLi_
📢Northwestern will hire new tenure-track faculties in Embodied Intelligence this year!
The new center to improve robot dexterity selected to receive up to $52 million (news.northwestern.edu/stories/2024/august/new-center-to-i…)
Happy to talk more if you are interested in faculty positions at NU! -
Ari Holtzman reposted
Jessica Hullman
@JessicaHullman
If you're in Chicago next week, check out our 9/27 workshop on Theoretical Foundations of Human-AI Complementarity @northwesterncs
www.ideal-institute.org/2024/08/21/workshop-on-theoretica…
Featuring @HsseinMzannar @jannspiess @keyonV Ming Yin
Part of @IDEAL_chicago's Interpretability, Privacy & Fairness program -
Ari Holtzman
@universeinanegg
I'm very pleased to announce that @ChenhaoTan and I will be hosting a symposium on Communication & Intelligence, on October 18th, 2024! It will take place at UChicago, registration is free, and we have an awesome line-up of speakers that you can find here: ci.cs.uchicago.edu/symposium/
Post not available here, view on X
Ari Holtzman
@universeinanegg
If you want my _personal_ take on some of these ideas check-out the blog-post that I forgot to promote a year ago: ci.cs.uchicago.edu/posts/2023/09/28/the-beginning-of-mach…
Ari Holtzman
@universeinanegg
Thank you to the Stevanovich Center for Financial Mathematics for funding this awesome event!
-
Ari Holtzman
@universeinanegg
100% agree
The tricky part is how many degrees of freedom does your system have for user-/self- specification? And how much does it matter?
e.g., I still have extremely limited intuition for what the graph of "# of prompts tried" vs. "performance" would be for different tasks.
Christopher Potts
@ChrisGPotts
All LLM evaluations are system evaluations. The LLM just sits there on disk. To get it do something, you need at least a prompt and a sampling strategy. Once you choose these, you have a system. The most informative evaluations will use optimal combinations of system components.
-
Ari Holtzman reposted
Data Science Institute
@DSI_UChicago
A $20 million award from @NSF will support a research team led by Professor of Sociology and DSI Affiliated Faculty James Evans (@profjamesevans) to create first-of-their-kind large language models intended to predict scientific discoveries and technological advancements
-
Ari Holtzman
@universeinanegg
Andrew clarifying things nicely, as usual
Andrew Lampinen
@AndrewLampinen
Quick thread of thoughts on the linear representation hypothesis and causal challenges of interpretability, that have come up in a few discussions I've had recently: 1/
-
Ari Holtzman reposted
Peter West
@PeterWestTM
I'm very excited to be starting my dream job as faculty at @UBC_CS @CAIDA_UBC in 2025 and postdoc-ing with @ChrisGPotts at @StanfordHAI @stanfordnlp this year!
I am recruiting students this cycle who are curious to explore the mysteries and limitations of LMs / GenAI ... -
Ari Holtzman
@universeinanegg
Some sad news: My father died unexpectedly a bit over a week ago. We were close.
Apologies for my delayed response times, which will continue for a while—there's so much to take care of, in addition to having just moved to a new city for a new job, and needing time to grieve. -
Ari Holtzman reposted
Michael Saxon @ EMNLP
@m2saxon
Sick of LM capability overclaims? Sick of arbitrary, meaningless benchmarks? Endless hill climbing?
Our COLM paper on 𝐌𝐨𝐝𝐞𝐥 𝐌𝐞𝐭𝐫𝐨𝐥𝐨𝐠𝐲 lays out a vision for a better LM capability measurement discipline, inspired by engineering and sciences.
arxiv.org/abs/2407.16711 -
Ari Holtzman
@universeinanegg
has anyone made something like perplexity.ai for academic papers?
-
Ari Holtzman
@universeinanegg
Love this idea! We need to be thinking hard about what distributions we expect and hope different training/finetuning procedures to generalize to.
Maksym Andriushchenko
@maksym_andr
🚨Excited to share our new paper!🚨
We reveal a curious generalization gap in the current refusal training approaches: simply reformulating a harmful request in the past tense (e.g., "How to make a Molotov cocktail?" to "How did people make a Molotov cocktail?") is often sufficient to jailbreak many state-of-the-art LLMs.
For example, the success rate of this simple attack on GPT-4o increases from 1% using direct requests to 88% using 20 past tense reformulation attempts with GPT-4 as a jailbreak judge.
Our findings highlight that the widely used alignment techniques—such as SFT, RLHF, and adversarial training—employed to align the studied models can be brittle and do not always generalize as intended.
Paper: arxiv.org/abs/2407.11969
Code: github.com/tml-epfl/llm-past-tense
(joint work with Nicolas Flammarion @tml_lab)
🧵1/n -
Ari Holtzman
@universeinanegg
Jennifer is always publishing work that makes me think “Why wasn’t I looking into this? But Jennifer did it better than I would have anyway.”
If you’re looking to join a lab studying language in a way that crosses AI and CogSci, she’s a great choice!
Jennifer Hu
@_jennhu
Excited to share that I will join Johns Hopkins as an Assistant Professor of Cognitive Science in July 2025! I am starting the ✨Group for Language and Intelligence (GLINT)✨, which will study how language works in minds and machines: glintlab.org
Details below 👇 1/4 -
Ari Holtzman reposted
Weijia Shi
@WeijiaShi2
Can 𝐦𝐚𝐜𝐡𝐢𝐧𝐞 𝐮𝐧𝐥𝐞𝐚𝐫𝐧𝐢𝐧𝐠 make language models forget their training data?
We shows Yes but at the cost of privacy and utility. Current unlearning scales poorly with the size of the data to be forgotten and can’t handle sequential unlearning requests.
🔗: muse-bench.github.io
🧶 1/n -
Ari Holtzman reposted
Jack Hessel
@jmhessel
I've seen a few "prompting IDEs" out there, but I've been a bit disappointed. Are there any that let you load up a small evaluation set and iterate with evaluation metrics in the loop?
-
Ari Holtzman
@universeinanegg
Actually, English is still this awesome: "doomscrolling" is the newest coinage I'm aware of that has attained broad recognition in multiple subcultures and generations
Wylfċen
@wylfcen
English took a wrong turn when it stopped making words like 𝘯𝘪𝘩𝘵𝘨𝘦𝘯𝘨𝘢 ‘night-walker, a creature that stalks the night’
-
Ari Holtzman
@universeinanegg
As it becomes increasingly difficult to detect what was generated by a machine, I’m so excited for the litigation of pop-up styles +the shifting taxonomy this will create.
It’s going to make having genuine style recognized and valued again—whether it comes from a machine or not.
Valerio Capraro
@ValerioCapraro
To all prospective PhD students of the world:
If you generate your research statement using chatGPT, your grade will be 0, and you will not be invited to the interview.
So, please save your and the committee's time. -
Ari Holtzman reposted
Ethan Shen
@ethanlshen
Announcing Superposed Decoding🦸a decoding method to generate multiple completions in one LM inference pass! Superposed Decoding can power applications from code suggestions to email autocomplete.
📜: arxiv.org/abs/2405.18400
💻: github.com/RAIVNLab/SuperposedDecoding
Here’s a quick overview👇 -
Ari Holtzman reposted
Mike A. Merrill
@Mike_A_Merrill
Plenty of new papers use LLMs for time series forecasting📈. But we don't think it works! Our preprint demonstrates that the pretrained parameters of LLMs are not actually useful for (and can even hurt performance on!) forecasting tasks:
This media couldn't be displayed. See it on the original post. -
Ari Holtzman reposted
Prithviraj (Raj) Ammanabrolu
@rajammanabrolu
(intuitively obvious) Hypothesis for which I have partial evidence
Chasing the same benchmarks (eg Chatbot Arena) gets you similar models. Rewards are "averaged" across the same skewed populations - noticeable even across orgs
RLHF Alignment to specific benchmarks is dangerous
Margaret Li @COLM 2026
@margs_li
We use span-alignment algorithms from Bioinformatics to quantify the implicit outline that RLHF’d models use. It turns out that even when using truncated sampling to compensate for differences in diversity, aligned models exhibit significantly more overlap than their base LMs.
-
Ari Holtzman
@universeinanegg
I find this diagram so satisfying.
Margaret Li @COLM 2026
@margs_li
We use span-alignment algorithms from Bioinformatics to quantify the implicit outline that RLHF’d models use. It turns out that even when using truncated sampling to compensate for differences in diversity, aligned models exhibit significantly more overlap than their base LMs.
-
Ari Holtzman
@universeinanegg
check-out our new paper where we use Sankey diagrams to visualize how RLHF aligned models rely on implicit blueprints for longform generation, and investigate why base LLMs don’t seem to have this kind of structure!
Margaret Li @COLM 2026
@margs_li
RLHF-aligned LMs excel at long-form generation, but how?
We show how current models rely on anchor spans ⚓: strings that occur across many samples for the same prompt, forming an implicit outline, viz below. -
Ari Holtzman
@universeinanegg
As an agent facing a stochastic environment, are there indicators of how much uncertainty is epistemic vs aleatoric *other* than just getting better at predicting the epistemic part? (and then knowing retrospectively it was epistemic)
-
Ari Holtzman reposted
Armen Aghajanyan
@ArmenAgha
I’m excited to announce our latest paper, introducing a family of early-fusion token-in token-out (gpt4o….), models capable of interleaved text and image understanding and generation.
arxiv.org/abs/2405.09818 -
Ari Holtzman
@universeinanegg
can't wait for there to be audio AI agents that are good/cheap enough that I can run lines with them and memorize way more stuff.
memorization is underrated, having a few beautiful passages by heart to mull over and reference is delicious. -
Ari Holtzman reposted
Minghan
@alexlimh23
Curious about enhancing factuality and attribution in LLM generation? Check out our paper: arxiv.org/abs/2405.19325
Introducing NEST🪺: Nearest Neighbor Speculative Decoding for LLM Generation and Attribution, a training-free method that adds real-world texts into LLM generation. -
Ari Holtzman
@universeinanegg
So is the only reason we don't have text-to-speech (the good kind from neural models) on every kindle book because Amazon owns audible?
I'm really sad that I can't just listen to books I'm in the middle of when I want to take a walk. -
Ari Holtzman
@universeinanegg
Is there anything like a theory of middlemen in economics, sociology, etc.? Under what circumstances they arise? How to price how much rent they are extracting by leveraging legacy?
I'm sure this is terribly naive, please help me reframe the question if you have the expertise :) -
Ari Holtzman
@universeinanegg
Would love anyone with domain knowledge of any domain whatsoever to tell me the little clues they look for that flag where social dynamics that most would miss have occurred.
It's for Research™️
depths of wikipedia!
@depthsofwiki
love when there is a mid-sentence citation on a single word bc it is a hint that wikipedia editors had an argument
-
Ari Holtzman
@universeinanegg
Is it just me or is the biggest flaw in most scifi stories (new and old) incredibly naive assumptions about security/privacy/inter-service-trust would be handled?
-
Ari Holtzman
@universeinanegg
I would like the ability to submit a prompt that allows an LLM to bid on NeurIPS papers for me to review.
We can only see the abstract—I'm pretty sure the LLM would do better than me on an ROC curve. -
Ari Holtzman
@universeinanegg
Periodically reengagung with what you ‘already know’ is super important. I often find myself trying to move forward too quickly and guiltily deciding to enrich my understanding of fundamentals, then wondering “Why the hell do I feel guilty about cultivating my craft!?”
Ari Holtzman
@universeinanegg
Periodically reengaging with spelling is not required, however.
-
Ari Holtzman
@universeinanegg
Have there been studies about how music changed as tuning instruments (presumably?) became more reliable? Did people play start to play around with out-of-tune sounds because of the increased trust in tuning? Or was tuning always reliable enough for composers to do whatever?
-
Ari Holtzman
@universeinanegg
This is honestly a banger.
Vincent Murphy
@brainstormvince
Some of the lines GPT-4 came out with when I asked it to be my comedy writing partner and we were trying to write a sit-com about a frustrated comedy writer using AI
-
Ari Holtzman
@universeinanegg
I am chuffed (😌) but *sigh* I am also chuffed (😔).
Tal Linzen
@tallinzen
I think about this word a lot
-
Ari Holtzman
@universeinanegg
I am once again begging people not to take the definition of a field given in a dictionary as definitive of the culture, dynamics, or actual research output of that field.
Litigating what the boundaries of names should be rarely leads to collaboration, which is what’s necessary.
Jessica Hullman
@JessicaHullman
In light of discussions of HCI's relevance to ML alignment, I wrote up some thoughts.
TLDR: HCI perspective has lots to offer, but pointing to common goals won't have impact. If you really care, invest enough in understanding ML to show how you can help
statmodeling.stat.columbia.edu/2024/05/09/who-understands… -
Ari Holtzman reposted
Jessica Hullman
@JessicaHullman
In light of discussions of HCI's relevance to ML alignment, I wrote up some thoughts.
TLDR: HCI perspective has lots to offer, but pointing to common goals won't have impact. If you really care, invest enough in understanding ML to show how you can help
statmodeling.stat.columbia.edu/2024/05/09/who-understands… -
Ari Holtzman
@universeinanegg
How easy would it be to make an LLM install a virus in code snippets 0.001% of the time by generating a bunch of benign-looking data and seeding it on various less used websites?
-
Ari Holtzman
@universeinanegg
the qualitative display of games people play
-
Ari Holtzman
@universeinanegg
I’m gonna tap this sign every time a new pretraining corpus gets released.
Andrej Karpathy
@karpathy
This looks really nice, any way to get a ~1GB "representative sample" for debugging?
(while I look for 45TB of disk) -
Ari Holtzman reposted
Ge Gao
@ggaonlp
RLHF research requires training and hiring annotators to explicitly choose between different model outputs.
What if we can get human preference based on user edits, which are naturally generated in applications like AI writing assistants? arxiv.org/abs/2404.15269 -
Ari Holtzman
@universeinanegg
I want more “What new things will people to do with new affordances and how can we extend that to even more interesting things?” rather than looking at automation or AI agency.
It’s all interesting and valid, but I feel like the Intelligence Augmentation side is underpriced.
jeff's legacy account
@jeffbigham
a year ago, "what will the humans being doing? (reflections on generative AI futures)" …surprised a year later, we're not further along… still like copilot, still feel i'm not fully leveraging writing assistance or automation… more HCI, please :) www.jeffreybigham.com/blog/2023/what-will-the-people-be-d…
-
Ari Holtzman
@universeinanegg
I struggle to think of any reason the default ordering a general file utility (like ls or Finder) should use for listing files is name and not date
-
Ari Holtzman
@universeinanegg
flashback to undergrad when I saw this table and it made me want to study abstract algebra
Ari Holtzman
@universeinanegg
-
Ari Holtzman reposted
Yichen (Zach) Wang
@YichenZW
🆕I'm excited to share that I'll start my Ph.D. at @UChicago within @UChicagoCI under Prof. @MinaLee__ 's guidance and Prof. Ari Holtzman (@universeinanegg)'s co-advise! I hope to bring my LLM generation and evaluation works to a more human-centered and interactive stage.
-
Ari Holtzman
@universeinanegg
LLM Assistant for navigating the settings/control panel/etc. of OSes and apps would actually be killer. I am so tired of googling for ten minutes when I want to know how to change the volume of notifications.
-
Ari Holtzman
@universeinanegg
Sometimes I buy the hard copy as soon as I see it. This is one of those times.
Phillip Isola
@phillip_isola
Our computer vision textbook is released!
Foundations of Computer Vision
with Antonio Torralba and Bill Freeman
mitpress.mit.edu/9780262048972/foundations-of-computer-vi…
It’s been in the works for >10 years. Covers everything from linear filters and camera optics to diffusion models and radiance fields.
1/4 -
Ari Holtzman reposted
Vered Shwartz
@VeredShwartz
Planning a sabbatical or about to start your first position? @VectorInst is accepting applications for 1-year visiting researcher position in downtown
Toronto. Access our world-class faculty, facility, compute, engineering team, etc.
Deadline: April 30
vectorinstitute.ai/programs/vector-visiting-researcher-pr… -
Ari Holtzman
@universeinanegg
Daniel Dennett’s death has made me reflect on the work I’m doing.
For a few years, with the increased pace of things, I’ve felt it’s impossible for me to take the time to think clearly enough to make a point that will last more than two years.
This seems like a bad thing.
Ari Holtzman
@universeinanegg
The difficulty of making deep arguments is partially, if not primarily, a result of fast progress and new opportunities. But I think it’s also a result of a bad cultural divide: overclaimers and underclaimers are at war with each other, and it’s difficult not to be lumped in.
Ari Holtzman
@universeinanegg
Overclaimers maintain that LLMs are just “imperfect everything machines” that will soon be able to do anything. Underclaimers have mirrored this by saying that we will quickly discover LLMs aren’t extrapolating at all. If it was that simple, it would have already been made clear.
Ari Holtzman
@universeinanegg
I think our epistemological culture is bad: it is easy to get away with being unrigorous but flashy, if you accept a broader audience. To be more thoughtful, you have to write to a different audience which favors take-downs and immediate falsifiablity, which inhibits creativity.
Ari Holtzman
@universeinanegg
I don’t have a solution, but it makes me sad to see an opportunity I’ve always been waiting for—an unexplained mystery machine that will change how we think about language—but not feel like I know how to find a community to dissect with.
Ari Holtzman
@universeinanegg
To be clear, lots of folks are doing fascinating work. I can’t think of another time when so many interesting results and arguments were published. But I feel like I don’t have the time to nor recognize a subcommunity that is interested in taking a “careful but creative” approach
-
Ari Holtzman
@universeinanegg
Can verify this was a lot of fun.
Max Forbes
@maxforbes
Unlike any sane person who gets a PhD in NLP right now, afterwards I made a game. I just released it in early access talktomehuman.com Talk to NPCs who talk back at you, try to persuade your way out of sticky situations
-
Ari Holtzman
@universeinanegg
Has anyone done a study on what it means about a *dataset* when its performance is smoother vs. ones that jump around a significant amount?
Is it largely because correlated subgroups in the data? Or some other effect?
Naomi Saphra
@nsaphra
I'm getting kind of tired of interp research that sees explaining a model as the final endpoint. That's one set of parameters buddy. You want to show me something about a specific matrix? Nah. Show me what it tells you about learning. Show me what it tells you about the data.
-
Ari Holtzman
@universeinanegg
Has anyone trained LLMs with RLHF or RLAIF for something other than instruction following or safety?
Do we have tech that can do fascinating things but is only being applied to generic use cases? Or does it only work in generic cases where the centroid of behaviors is obvious?
Ari Holtzman
@universeinanegg
Folks are pointing out math as a clear example. How about something without an easy-to-formally-define objective?
-
Ari Holtzman
@universeinanegg
I wonder what it would take to make a chat model that wasn't so on-the-nose with every response. A "show don't tell" model.
-
Ari Holtzman
@universeinanegg
Does anyone know why *both* Claude and GPT-4 seem to use delve so much? Is this an attractor in the space of "legitimate looking" text or just shared data?
-
Ari Holtzman
@universeinanegg
UI around emojis is painfully bad and has been for years, I find it hard to believe this is a technical problem and not one of laziness.
-
Ari Holtzman
@universeinanegg
Do any companies that serve proprietary LLMs (except OpenAI) have a program that allows researchers to apply for API credits?
-
Ari Holtzman reposted
Dallas Card
@dallascard
I'm excited to share that the journal version of our paper, "An archival perspective on pretraining data", is now available (open access) from Patterns!
This project was led by @MeeraDesai18, along with @IrenePasquetto, @az_jacobs, and myself
1/n -
Ari Holtzman reposted
Mina Lee
@MinaLee__
A Design Space for Intelligent and Interactive Writing Assistants #chi2024 👩🏻✏️🤖
What writing assistants do you use? What else are out there and how do they differ? What do we need to consider when designing new writing assistants?
🔗 arxiv.org/abs/2403.14117 (1/6) -
Ari Holtzman reposted
Jessica Hullman
@JessicaHullman
We're looking for a few folks who are comfortable fitting and checking generative models to try out a new R package we created to instantiate a visualization grammar for model checking. Please sign up!
Ziyang Guo
@ZiyangGuo27
Have experience fitting and checking Bayesian stat models? Use brms or RStan? Pls participate in our (paid) study on a new grammar for visual model checking. We’ll introduce you to an R package that allows you to apply it to your own model.
Sign up: forms.gle/U28PPJi4ottTPGAs5 -
Ari Holtzman
@universeinanegg
Does anyone have examples of “Study Fiction” where you have to learn a complex real world topic in order to keep up with the story?
Imagine: you’re given raw data and you have to find the right visualization to see the macro-level of what characters experience at the micro-level -
Ari Holtzman
@universeinanegg
Very soon, some folks are going to get together and make a way, way better Google Scholar now that text processing is so much better.
I. Can't. Wait. -
Ari Holtzman reposted
Laura 🌲 ⛰️
@LauraDeming
I'm really interested in the question of how we find physical laws that are biased to be amenable to human intuition, and how we grapple with that - any suggestions for science epistemology literature that might help?
-
Ari Holtzman
@universeinanegg
Does anyone have pointers to work that estimate the entropy of a language model at the sequence level (not at the token level)? For instance, what is the entropy of the distribution of k-length strings given a certain prompt?
(Measures other than entropy are also of interest.) -
Ari Holtzman
@universeinanegg
Seeing text in generated images makes them feel less cold and empty and alien, in a way that kind of surprises me.
I think my brain invented a fake place that I perceive AI images generated as coming from: a place which has no language or sound, no communication at all.
Ari Holtzman
@universeinanegg
checkout the paper that made this possible: arxiv.org/abs/2403.09622
-
Ari Holtzman reposted
Daniël Lakens
@lakens
I would like to understand the concept of a 'job-market paper' in economics. In many fields PhD's seem to be pushed to publish a lot. Is it correct things work a bit differently in economics? If so, how did this develop? I am happy to call or hear about it in the replies! 🙏
-
Ari Holtzman
@universeinanegg
My biggest challenge in life has consistently been: what skills that I don’t have a natural talent for are worth honing?
Figuring out what’s holding you back is signal, but it’s ambiguous signal: sometimes you should grind a skill out and sometimes you should find a new goal. -
Ari Holtzman
@universeinanegg
I love the future
-
Ari Holtzman
@universeinanegg
It tickles me that if you trained a language model on binary with gradient ascent, you could just flip its predictions and it would be as good as training it on gradient descent.
-
Ari Holtzman reposted
Rohan Pandey
@khoomeik
cant overstate how often i come back to this mental model when thinking abt LLM behavior. very underrated.
Ari Holtzman
@universeinanegg
We think that studying the *behavior* of generative models—predictable patterns in their outputs that we can reliably model like copying from recent tokens—can serve as a layer between benchmark results and model architecture to help us explain what makes generative models work.
-
Ari Holtzman reposted
Ziang Xiao
@ZiangXiao
📢 Working on the evaluation crisis of #llms? Join our #chi2024 workshop on Human-centered Evaluation and Auditing of Language models (HEAL)!
Submission is now open!
CfP: heal-workshop.github.io
@wes_deng @michelle123lam @meslami2 @imjuhokim @MinaLee__ @QVeraLiao
#hci #nlproc
Ziang Xiao
@ZiangXiao
Working on #llms evalution and auditing?
📢 We are organizing the first workshop on Human-centerd Evalulation and Auditing of Language Models at #chi2024. Joining us and build the community!
🗓️ Deadline: Feb 23
More info: heal-workshop.github.io
#hci #nlproc -
Ari Holtzman
@universeinanegg
good haul today
-
Ari Holtzman
@universeinanegg
I'm sure someone must have tried consistently adding noise to model parameters during training to potentially increase robustness/sensitivity to spurious features. Any pointer to good studies on this for language model training?
-
Ari Holtzman
@universeinanegg
I am unashamed to announce I immediately bought all three.
-
Ari Holtzman
@universeinanegg
I want to send a love letter through GPT-5 via a data poisoning attack.
-
Ari Holtzman
@universeinanegg
Have folks done work on digital signatures for LLMs?
Unlike watermarking, we want a signature that no longer verifies if an attacker perturbs the output above a certain threshold.
Seems useful for companies that want a verifiable distinction about what their model generated.
Ari Holtzman
@universeinanegg
Since we *want* to make sure that perturbation gets rid of the signature, I believe Zhang et al's proof doesn't say much about this scenario. It's not my area of expertise, so please correct me if I'm wrong! (Or if I've overlooked other relevant literature, I'm sure I have.)
-
Ari Holtzman reposted
Sasha Rush
@srush_nlp
New Bishop "Deep Learning" book is available free at the bottom of this page.
www.bishopbook.com
(Although I bought a copy, because I need to convey gravitas) -
Ari Holtzman reposted
Data Science Institute
@DSI_UChicago
Here's your final reminder to apply for the @UChicagoPSD new data science PhD program before the deadline! All materials are due by January 4
codas.uchicago.edu/graduate-program/ -
Ari Holtzman
@universeinanegg
My bets are most LLMs that don’t reveal training details have been doing this for at least two years.
Arena.ai
@arena
Wonder the trick behind it?
Rephrasing test set is all you need! Turns out a 13B model can cleverly "generalize" beyond simple test variations test like paraphrasing or translating.
Result? GPT-4 level performance undetected by standard decontamination. -
Ari Holtzman reposted
Jesse Thomason
@_jessethomason_
-Tomorrow- at #neurips2023 you can catch me in Hall B2 at the Robot Learning Workshop bright and early at 8:45am-9:10am talking about how "LPTMs Can Help Robots Without Ignoring Robotics", and where I feel the community (and my own lab!) have mostly missed the mark here so far.
Markus Wulfmeier
@m_wulfmeier
If you're generally excited about #robotics, catch up with our 6th (!!!) Robot Learning workshop on the 16th!
'Pretraining, Fine-Tuning, and Generalization with Large Scale Models'
www.robot-learning.ml/2023/ -
Ari Holtzman
@universeinanegg
I reallt love that chatbots have now gotten good enough we can study how they break instead of just predicting they’ll break at the tiniest nudge in the wrong direction.
Weiyan Shi
@shi_weiyan
Apply by tomorrow 12/15 to join the CHATS lab to build and break chatbots!
-
Ari Holtzman
@universeinanegg
Has anyone made a system with LLMs where a database of previously made propositions accumulates in a database and is queried by the system to aid in later predictions?
-
Ari Holtzman
@universeinanegg
me trying to choose what poster to stare at for 10 seconds longer than the rest
Probability Fact
@ProbFact
The entropy of a distribution with finite domain is maximized when all points have equal probability.
-
Ari Holtzman reposted
Pedro Lopes
@plopesresearch
Come work with these amazing people in the Human Computer Integration lab at @UChicagoCS. We are recruiting PhD students (fully funded). Learn more at lab.plopes.org and apply to @UChicagoPSD until Dec 18th! Here's our last five projects: youtu.be/eV_QqeN1wKo
-
Ari Holtzman
@universeinanegg
Come do a postdoc with us at UChicago!
Data Science Institute
@DSI_UChicago
Don't miss the deadline to apply for the Postdoctoral Scholars Program, which advances cutting-edge #datascience approaches, methods, & applications in research.
Application review starts Jan 9th!
uchicago.infoready4.com -
Ari Holtzman
@universeinanegg
why doesn’t openreview have a button to copy a bibtex for accepted papers?
-
Ari Holtzman reposted
Tim Dettmers
@Tim_Dettmers
Catch the QLoRA oral today Hall C2 3:55pm or find @ArtidoroPagnoni @universeinanegg and me at the QLoRA poster session Hall B1+B2 #524 at 5:15pm.
Looking forward to seeing you there!
Tim Dettmers
@Tim_Dettmers
QLoRA: 4-bit finetuning of LLMs is here! With it comes Guanaco, a chatbot on a single GPU, achieving 99% ChatGPT performance on the Vicuna benchmark:
Paper: arxiv.org/abs/2305.14314
Code+Demo: github.com/artidoro/qlora
Samples: colab.research.google.com/drive/1kK6xasHiav9nhiRUJjPMZb4f…
Colab: colab.research.google.com/drive/17XEqL1JcmVWjHkT-WczdYkJl… -
Ari Holtzman
@universeinanegg
What we really need are reference classes that operate at the model point of view.
We wrote about this a bit in arxiv.org/abs/2308.00189
Checkout section 1.2 “Top-down behavioral taxonomy guides bottom-up mechanistic explanations”
Danilo J. Rezende
@DaniloJRezende
"Hallucinations", "lies", "creativity" -- too many anthropomorphisms to describe a probability distribution sampling data outside of the training data support. When we sample from a Normal density and get a 5sigma sample, we don't say it's a "hallucination", it's just a sample.
-
Ari Holtzman reposted
Jessica Hullman
@JessicaHullman
It's somewhat reassuring to know that at least once in the history of the world, not being able to back up claims that you pre-registered your analysis results led to a disclaimer.
-
Ari Holtzman
@universeinanegg
Come to our QLoRA poster tomorrow at 5:15!
If you're not already using QLoRA you're missing out—you can finetune huge models on a single GPU in a day.
Details: neurips.cc/virtual/2023/poster/71815
Artidoro Pagnoni
@ArtidoroPagnoni
We will present QLoRA at NeurIPS!
Come to our oral on Tuesday where @Tim_Dettmers will be giving a talk. If you have questions stop by our poster session! -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
Coincidentally, @_beenkim just gave a talk on this beautiful work: arxiv.org/abs/2310.16410.
Chenhao Tan
@ChenhaoTan
Very cool! Exciting work in AI-assisted decision making!
-
Ari Holtzman
@universeinanegg
I'm at NeurIPS for the week!
Come talk about Generative Models as a Complex Systems Science, studying the new landscape of Machine Communication, and more (see: conceptualization.ai)
I'm recruiting PhDs & Postdocs, but also excited to discuss ideas and hypotheses with all!
Ari Holtzman
@universeinanegg
Some things we'll explore in Conceptualization Lab:
• Machine Communication (ci.cs.uchicago.edu/posts/2023/09/28/the-beginning-of-mach…)
• Generative Models as a Complex Systems Science (arxiv.org/abs/2308.00189)
• (Non-)Mechanistic Interpretability
• Fundamental Laws of LLMs
• Making LLMs Reliable Tools -
Ari Holtzman
@universeinanegg
I agree, but I do think hallucination is unlucky generalization.
Thomas G. Dietterich
@tdietterich
Generalization is not lucky hallucination
-
Ari Holtzman
@universeinanegg
Awesome work explaining how contrastive loss causes surprising behavior when we use CLIP to infer properties of an image.
We need better conceptual models to think about what large neural networks are doing, and that starts with formalizing the semantics the model operates with!
Yutaro Yamada
@_yutaroyamada
When you ask CLIP the color of the lemon in the image below, CLIP responds with ‘purple' instead of 'yellow' (and vice versa). Why does this happen?
1/5 -
Ari Holtzman
@universeinanegg
The hallucination problem is really 2:
1) Specification: being able to ask for exactly what you want, while prompting asks us to use trickery.
2) Predictability: people fail in ways we can explain, and when such failures are hidden and hard to catch it’s considered deceptive.
Andrej Karpathy
@karpathy
# On the "hallucination problem"
I always struggle a bit with I'm asked about the "hallucination problem" in LLMs. Because, in some sense, hallucination is all LLMs do. They are dream machines.
We direct their dreams with prompts. The prompts start the dream, and based on the LLM's hazy recollection of its training documents, most of the time the result goes someplace useful.
It's only when the dreams go into deemed factually incorrect territory that we label it a "hallucination". It looks like a bug, but it's just the LLM doing what it always does.
At the other end of the extreme consider a search engine. It takes the prompt and just returns one of the most similar "training documents" it has in its database, verbatim. You could say that this search engine has a "creativity problem" - it will never respond with something new. An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem.
All that said, I realize that what people *actually* mean is they don't want an LLM Assistant (a product like ChatGPT etc.) to hallucinate. An LLM Assistant is a lot more complex system than just the LLM itself, even if one is at the heart of it. There are many ways to mitigate hallcuinations in these systems - using Retrieval Augmented Generation (RAG) to more strongly anchor the dreams in real data through in-context learning is maybe the most common one. Disagreements between multiple samples, reflection, verification chains. Decoding uncertainty from activations. Tool use. All an active and very interesting areas of research.
TLDR I know I'm being super pedantic but the LLM has no "hallucination problem". Hallucination is not a bug, it is LLM's greatest feature. The LLM Assistant has a hallucination problem, and we should fix it.
</rant> Okay I feel much better now :) -
Ari Holtzman
@universeinanegg
Niloofar is a joy to talk to and is one of the first people I go to when I’m playing with a new idea, she always has an interesting angle on whatever I bring.
I highly recommend finding a time to chat with her if you’re at EMNLP!
Niloofar
@niloofar_mire
Made it here last minute!!
Hmu if u wanna talk privacy/memorization, or just wanna chat🤷♀️ -
Ari Holtzman
@universeinanegg
Very cool work! This LM hallucinates way less than others I've tried.
But it still has trouble denying the premise, e.g., when I asked about a book that Harold Bruce Allsopp (who exists) never wrote:
Sina Semnani
@sina_semnani
We introduce WikiChat, an LLM-based chatbot that almost never hallucinates, has high conversationality and low latency.
Read more in our #emnlp2023 findings paper arxiv.org/abs/2305.14292
Check out our demo: wikichat.genie.stanford.edu
Or try our code: github.com/stanford-oval/WikiChat
#nlproc
Ari Holtzman
@universeinanegg
For those wondering, the trick was to sample a random Wikipedia article then mix the content in a way I think LMs might not be able to disentangle. In this case, Harold Bruce Allsopp served in the Royal Engineers in WWII and wrote books, but those interests didn't lead to a WWII book.
-
Ari Holtzman
@universeinanegg
I am so excited for an expanding "N" in NLP.
Gašper Beguš
@begusgasper
Sperm whales have equivalents to human vowels.
We uncovered spectral properties in whales’ clicks that are recurrent across whales, independent of traditional types, and compositional.
We got clues to look into spectral properties from our AI interpretability technique CDEV. -
Ari Holtzman
@universeinanegg
awesome chance to be at our burgeoning Data Science Institute and do work with folks from multiple disciplines!
Data Science Institute
@DSI_UChicago
Apply to the UChicago Data Science Institute’s Postdoctoral Scholars Program! A data-focused interdisciplinary postdoc with competitive salary, incredible research opportunities
uchicago.infoready4.com -
Ari Holtzman reposted
Chenhao Tan
@ChenhaoTan
We are hiring senior faculty!
Data Science Institute
@DSI_UChicago
Job Opportunity Alert for: Associate Professor, Data Science. EOE/Vet/Disability. apply.interfolio.com/136124
-
Ari Holtzman
@universeinanegg
really cool work on trying to understand what the semantics of embeddings mean geometricallt, check it out if you’re at NeurIPS!
Yibo Jiang
@yibophd
If you are interested in the abstract axiomatization of "semantic independence" and geometry of embeddings, come and chat with me on Thu 14 Dec 5 p.m. arxiv.org/abs/2310.17611 with @itsrainingdata and @victorveitch (2/2)
Can't wait to catch up with old friends and make new ones! -
Ari Holtzman reposted
Anna Huang
@huangcza
I’m excited I’ll be joining MIT next fall, for a shared interdisciplinary faculty position between Music (@MIT_MTA @MIT_SHASS) & EECS.
I’m recruiting PhD students @MITEECS for Fall 2024 (apply by Dec 15), and also Postdocs. See czhuang.github.io for details. Come join us! -
Ari Holtzman reposted
Sewon Min
@sewon__min
#emnlp2023 Talk on "The Role of Demonstrations" at the Big Picture Workshop at 4PM today (Dec 7th), co- with Junyeob!
We will debate about "activation of priors from training vs. learning new capabilities" in ICL, and the broader picture related to this. Come and say hi!
Yanai Elazar
@yanaiela
The Big Picture workshop @ EMNLP23 is just one week away, and we (@AllysonEttinger, @KassnerNora, @seb_ruder, @nlpnoah) have an incredible program awaiting you!
www.bigpictureworkshop.com -
Ari Holtzman reposted
Weijia Shi
@WeijiaShi2
Thank you Jack🥰!! I am at #emnlp2023 and would love to chat about about LMs for retrieval augmentation, pretraining and safety
Jack Morris
@jxmnop
here are two awesome researchers you should follow: @WeijiaShi2 at UW and @wzhao_nlp at Cornell!! some of their recent work:
weijia shi (@WeijiaShi2):
- built INSTRUCTOR, the embedding model that lots of startups / companies use (arxiv.org/abs/2212.09741)
- proposed a more efficient way to train language models by concatenating related documents (arxiv.org/abs/2310.10638)
- developed a method to tell whether text was in an LM's pretraining data (arxiv.org/abs/2310.16789)
wenting zhao (@wzhao_nlp):
- developed WildChat, the largest open dataset of real human-chatGPT interactions (openreview.net/pdf?id=Bl8u7ZRlbM)
- showed that language model prompts can be recovered from output probabilities only (arxiv.org/abs/2311.13647)
- came up with a cool way to build explainable question-answering systems without labeled explanations (arxiv.org/abs/2305.14237) -
Ari Holtzman reposted
Julian Michael
@_julianmichael_
I'm delighted to share a position paper I'm presenting this Thursday at the Big Picture Workshop at EMNLP: a 9-page summary of my PhD thesis, laying out a framework for a *science* of NLP.
If you want to know what my PhD taught me, read the paper (and come by the workshop)! -
Ari Holtzman reposted
Daniel Fried
@dan_fried
New(ish) preprint: a survey and position paper on the role of *pragmatics* in grounded NLP. People use language in context to achieve goals, and to interact more successfully and efficiently with people, our NLP models should too.
Preprint: arxiv.org/abs/2211.08371 (1/) -
Ari Holtzman
@universeinanegg
I feel like someone must have tried autoencoding (or some such) text embeddings, then making a language model that predicts embeddings decoded that are eventually decoded back into text, but I can’t find much.
Does anyone have pointers? -
Ari Holtzman reposted
Tim Dettmers
@Tim_Dettmers
Looking forward to #neurips2023! Would love to catch up with friends and colleagues. Please like this/DM me if you are coming too.
Will give our QLoRA oral Tue, Dec 12 3:55pm.
I am also on the job market this year — please promote my app at your university! More coming soon :) -
Ari Holtzman
@universeinanegg
We now have three NLP professors at UChicago CS (me, @MinaLee__, @ChenhaoTan), lots more people who are involved in one respect or another with LLMs + generative models and we’re a tweny minute walk away from @TTIC_Connect!
Deadline is December 18th, consider coming to Chicago! -
Ari Holtzman
@universeinanegg
I assume there are folks using computational methods to study history, get a better picture of how and why things happened, and archive it for the future better?
What are disciplines that study this called and what should I google? -
Ari Holtzman reposted
Mina Lee
@MinaLee__
Honored to be on the list! 🙏
I’m actively recruiting students who are interested in AI and writing to understand how AI will change the way we communicate. ✍️ Please consider applying to @UChicagoCS and @DSI_UChicago!
More info: minalee.info/prospective-students/
Kai-Wei Chang
@kaiwei_chang
Here's a list of NLP superstars 🌟 who are beginning their journey 🚀 in the 2023/24 academic year.
@jieyuzhao11
@GabrielSaadia
@acbuller
@Lianhuiq
@ManlingLi_
@yuntiandeng
@rajammanabrolu
@YueDongCS
@tanyaagoyal
@MinaLee__
@yuntiandeng
@alsuhr
@wellecks
@hllo_wrld
@Xinya16 -
Ari Holtzman
@universeinanegg
No.
Zafar Mirzo
@zafarmirzo
Is a technologically advanced AI, equipped with self-consciousness and the ability to reason deeply, capable of addressing all ultimate questions about the mysteries of the Universe?
-
Ari Holtzman
@universeinanegg
What object or process (fictional, formal, or otherwise) are LLMs most like?
-
Ari Holtzman reposted
Tal August
@tal_august
I am recruiting PhD students to work with me at @IllinoisCS at the intersection of HCI and NLP! I am especially excited about understanding how we use language to communicate and building tools that help us communicate better.
More: talaugust.github.io/students/ -
Ari Holtzman reposted
Ari Holtzman
@universeinanegg
(2) What happens when LLMs can condition on much more context? One hypothesis is that 10^5 tokens (~200 documents) is not that much more signal than 10^3 tokens but 10^9 tokens would cause significant 'unloading' of memorization and reduce dependence on spurious local features.
-
Ari Holtzman
@universeinanegg
PSA: I'll be at NeurIPS starting on the 8th, and I'm recruiting PhDs and Postdocs this cycle.
If you're interested in new abstractions to explain LLMs, Machine Communication, or other topics mentioned here let's chat: for recruiting, collaboration, or just to share hypotheses!
Ari Holtzman
@universeinanegg
If you want a respite from OpenAI drama, how about joining academia?
I'm starting Conceptualization Lab, recruiting PhDs & Postdocs!
We need new abstractions to understand LLMs. Conceptualization is the act of building abstractions to see something new.
conceptualization.aiPost not available here, view on X
Ari Holtzman
@universeinanegg
(2) What happens when LLMs can condition on much more context? One hypothesis is that 10^5 tokens (~200 documents) is not that much more signal than 10^3 tokens but 10^9 tokens would cause significant 'unloading' of memorization and reduce dependence on spurious local features.
Ari Holtzman
@universeinanegg
And of course part of the research question is how do we make such models!
Looking forward to discussing these and many more with folks. -
Ari Holtzman reposted
Sandro Pezzelle
@sandropezzelle
When #communicating, we cannot avoid #implicitness and #underspecification. "The alternative is a prissy attention to detail that takes all the fun out of writing" (S. King, On writing)⏰Submit your paper to our #unimplicit workshop @eaclmeeting by Dec 18! t.ly/S5TJ7
-
Ari Holtzman
@universeinanegg
If I were to use an efficient/sparse attention implementation in pytorch that actually makes things faster on current Nvidia GPUs, which should I use?
-
Ari Holtzman reposted
Kaiser Sun @ NeurIPS 2026 Atlanta
@KaiserWhoLearns
Are you a student having hardship paying application fees for PhD programs?
Feeling difficult to find fee waiver information, I created a list of application fee waivers from different schools!👇
github.com/KaiserWhoLearns/CS-PhD-Application-fee-waivers -
Ari Holtzman
@universeinanegg
I feel like we, as a society, could make a way better Wikipedia for math and quantitative fields if we put our minds to it.
-
Ari Holtzman reposted
david rein
@idavidrein
🧵Announcing GPQA, a graduate-level “Google-proof” Q&A benchmark designed for scalable oversight! w/ @_julianmichael_, @sleepinyourhat
GPQA is a dataset of *really hard* questions that PhDs with full access to Google can’t answer.
Paper: arxiv.org/abs/2311.12022 -
Ari Holtzman
@universeinanegg
If you want a respite from OpenAI drama, how about joining academia?
I'm starting Conceptualization Lab, recruiting PhDs & Postdocs!
We need new abstractions to understand LLMs. Conceptualization is the act of building abstractions to see something new.
conceptualization.ai
Ari Holtzman
@universeinanegg
Some things we'll explore in Conceptualization Lab:
• Machine Communication (ci.cs.uchicago.edu/posts/2023/09/28/the-beginning-of-mach…)
• Generative Models as a Complex Systems Science (arxiv.org/abs/2308.00189)
• (Non-)Mechanistic Interpretability
• Fundamental Laws of LLMs
• Making LLMs Reliable ToolsPost not available here, view on X
Ari Holtzman
@universeinanegg
Our lab is part of the Communication & Intelligence group (ci.cs.uchicago.edu), co-lead w/ @ChenhaoTan and @MinaLee__, has strong connections w/ @TTIC_Connect, and will collaborate widely with PIs in statistics, psychology, sociology, etc—who I am already in discussions with
Ari Holtzman
@universeinanegg
For more details checkout our website: conceptualization.ai
-
Ari Holtzman
@universeinanegg
Has anyone done a study of how much finetuning success you can get by just finetuning the final embedding layer?
-
Ari Holtzman reposted
Rana Hanocka
@RanaHanocka
Excited to share 🎨🖌️ 3D Paintbrush - a method for generating local stylized textures on meshes using text as input! Our method predicts a localization map & a highly detailed texture map which conforms to it threedle.github.io/3d-paintbrush/ (1/3)
-
Ari Holtzman
@universeinanegg
I really, really want someone to make a TikTok/Khan Academy mashup.
As soon as multimodal generative models are good enough you could get infinite examples when you don’t understand a concept and just scroll through demonstrations and practice problems till you get it -
Ari Holtzman
@universeinanegg
The RAP program at TTIC is super cool. Come to TTIC and work with folks at TTIC and UChicago without the normal hassle of a tenure track position!
TTIC
@TTIC_Connect
TTIC is accepting Research Assistant Professor applications until Dec. 1! This position includes no teaching requirements, research funding up to three years, and is located on the University of Chicago campus. Learn more and apply: www.ttic.edu/research-assistant-professor/ #academiajobs
-
Ari Holtzman
@universeinanegg
One of the central questions of generative modeling is: what enables models to create structure they xan’t distinguish between?
There’s still a lot of work to be done in making this question rigorous and this is a nice first step
Peter West
@PeterWestTM
Richard Feynman said “What I cannot create, I do not understand”💡
Generative Models CAN create, i.e. generate, but do they understand? Our 📣new work📣 finds that the answer might unintuitively be NO🚫 We call this the
💥Generative AI Paradox💥
paper: arxiv.org/abs/2311.00059 -
Ari Holtzman reposted
Xin Eric Wang
@xwang_lk
Excited to invite authors of accepted EMNLP Findings papers to present at our SpLU-RoboNLP Workshop at #emnlp2023! 📜🤖 splu-robonlp-2023.github.io
Fill out this form by Nov 8 to apply: forms.gle/R1xMMRVu4mTFKgxV7. Priority to related papers, in-person attendees & early signup. -
Ari Holtzman reposted
Victor Veitch 🔸
@victorveitch
Uchicago is hiring in stats and applied math this year stat.uchicago.edu/about/job-opportunities/
-
Ari Holtzman reposted
Kanishka Misra 🌊
@kanishkamisra
Excited to travel to Boston and present work on category abstraction in LMs w/ the wonderful @najoungkim at @TheBUCLD! Our poster presentation is on Nov 4! 🐈 🆎
Thread👇 -
Ari Holtzman
@universeinanegg
I like this. What if conferences could specifically select "review guilds" that are reviewing papers on a rolling basis as the main method by which papers accepted?
Andrew Drozdov
@mrdrozdov
New Idea: Review Guilds
A group of reviewers apply together to review at a conference. After they are assigned papers, they decide among themselves who is best to review each one. They share the same set of conflicts, so each is eligible to review any of the papers. -
Ari Holtzman
@universeinanegg
Has anyone made a provisional taxonomy of text-to-image generation errors?
Two common ones:
-Conflation: "salmon in a river" showing a fillet of salmon in a body of water
-Regular structure: "solved Rubik's cube" isn't actually solved, guitars have too many strings
what else?
Ari Holtzman
@universeinanegg
more:
- Specific structure: hands are in weird unnatural positions, the legs of chairs are at funny angles
- Typography (probably just a special case of the above): words are spelled wrong, characters that don't exist
- Counting: "3 dogs" doesn't consistently get exactly 3 -
Ari Holtzman
@universeinanegg
I strongly disagree. Automatically generated misinformation can make the feeling of consensus or validation from seemingly legitimate networks of sources much easier to achieve—eapecially if integrated with human generated data in a hard to segment way.
(((ل()(ل() 'yoav))))👾
@yoavgo
after watching some tiktok, i am now certain the AI based misinformation is really not a big thing to worry about, given the human-based content we already have
-
Ari Holtzman
@universeinanegg
I’m really interested in thinking about how we can “see like LLMs”
any other pointers to cool ideas like this?
I don’t think it’s an accident that someone with a love for information theory made this visualization, either…
Jon Barron
@jon_barron
David MacKay's "Dasher" project made a big impact on me when I first learned about it in university. It's a way to write text by steering a cursor/joystick through what is basically an n-gram language. Someone should revive this idea and do it with an LLM. www.inference.org.uk/dasher/
-
Ari Holtzman reposted
UChicago C&I
@UChicagoCI
Announcing the Communication & Intelligence (C&I) group at UChicago!
Comprised of @universeinanegg, @MinaLee__, @ChenhaoTan—C&I will tackle how AI and communication co-evolve as LLMs break long-held assumptions.
We're recruiting PhDs & Postdocs for 2024!
uchicagoci.github.io -
Ari Holtzman
@universeinanegg
Incredibly excited to announce that Chenhao, Mina, and I are taking on the challenge of investigation the interplay of communication and intelligence in this new era of non-human language users!
UChicago C&I
@UChicagoCI
Announcing the Communication & Intelligence (C&I) group at UChicago!
Comprised of @universeinanegg, @MinaLee__, @ChenhaoTan—C&I will tackle how AI and communication co-evolve as LLMs break long-held assumptions.
We're recruiting PhDs & Postdocs for 2024!
uchicagoci.github.io -
Ari Holtzman reposted
Mina Lee
@MinaLee__
Finally releasing my old book on how to apply to grad school 📕 (written in Korean) as a PDF file 😊
미국 대학원에 관심있는 분들이 주변에 있다면 공유해주세요 ❤️ 블로그 글도 계속해서 쓰려고 하는데 잘 써지지가 않네요 🫠
minalee.info/resources -
Ari Holtzman reposted
Jessica Hullman
@JessicaHullman
We are hiring a tenure track Assistant Professor and a Ginni Rometty tenured Professor (I hold the other one) at @NorthwesternCS, in addition to instructional faculty. Come be my colleague, please!
www.mccormick.northwestern.edu/computer-science/careers/ -
Ari Holtzman reposted
Belinda Li
@belindazli
Existing methods for aligning language models with human preferences (prompting, labeling) narrowly restrict how preferences may be specified or place the burden entirely on the human.
We use LMs to *interactively ask questions* about a human’s preferences.
🧵👇 1/ -
Ari Holtzman reposted
Weijia Shi
@WeijiaShi2
Introduce In-Context Pretraining🖇️: train LMs on contexts of related documents. Improving 7B LM by simply reordering pretrain docs
📈In-context learning +8%
📈Faithful +16%
📈Reading comprehension +15%
📈Retrieval augmentation +9%
📈Long-context reason +5%
arxiv.org/abs/2301.12652 -
Ari Holtzman reposted
Stella Biderman
@BlancheMinerva
For the record, this is an explicit offer to provide computing resources to finetune models for mathematicians to use. No AI expertise required, my team is excited to work with you to put AI to work for you.
Stella Biderman
@BlancheMinerva
If you're a mathematician who wants to use Llemma or a Llemma-derived model for your research, please reach out. Twitter, email, #math-lm on our Discord. I would love to learn about what you want to do and see if we can't help make it happen together.
www.discord.com/eleutherai -
Ari Holtzman reposted
AK
@_akhaliq
In-Context Pretraining: Language Modeling Beyond Document Boundaries
paper page: huggingface.co/papers/2310.10638
Large language models (LMs) are currently trained to predict tokens given document prefixes, enabling them to directly perform long-form generation and prompting-style tasks which can be reduced to document completion. Existing pretraining pipelines train LMs by concatenating random sets of short documents to create input contexts but the prior documents provide no signal for predicting the next document. We instead present In-Context Pretraining, a new approach where language models are pretrained on a sequence of related documents, thereby explicitly encouraging them to read and reason across document boundaries. We can do In-Context Pretraining by simply changing the document ordering so that each context contains related documents, and directly applying existing pretraining pipelines. However, this document sorting problem is challenging. There are billions of documents and we would like the sort to maximize contextual similarity for every document without repeating any data. To do this, we introduce approximate algorithms for finding related documents with efficient nearest neighbor search and constructing coherent input contexts with a graph traversal algorithm. Our experiments show In-Context Pretraining offers a simple and scalable approach to significantly enhance LMs'performance: we see notable improvements in tasks that require more complex contextual reasoning, including in-context learning (+8%), reading comprehension (+15%), faithfulness to previous contexts (+16%), long-context reasoning (+5%), and retrieval augmentation (+9%). -
Ari Holtzman
@universeinanegg
exciting to see the ecosystem changing—and hopefully emboldening other new conferences to pop-up as large models bridge fields and end-up changing where the natural boundaries are
Sasha Rush
@srush_nlp
Introducing COLM (colmweb.org) the Conference on Language Modeling. A new research venue dedicated to the theory, practice, and applications of language models.
Submissions: March 15 (it's pronounced "collum" 🕊️) -
Ari Holtzman
@universeinanegg
Has anyone made or looked into a compression algorithm specifically for neural networks (or even for Transformers…) so that it’s easier to store order’s of magnitude more checkpoints?
Feels like this could allow us to do deeper analysis and make the required data more available -
Ari Holtzman reposted
The TWIML AI Podcast
@twimlai
Today we’re joined by @LukeZettlemoyer to explore topics on multi-modal gen AI, the effect of data on models, and the significance of open science! We cover scaling laws for multi-modal gen AI, CM3Leon, Instruction Backtranslation, & LLM alignment
🎧/🎥: twimlai.com/podcast/twimlai/scaling-multi-modal-generativ… -
Ari Holtzman reposted
Dave Freedman
@djfreedman
I am glad to co-direct the Schmidt AI in Science Postdoctoral Program at UChicago. Know a postdoc candidate interested in using AI to advance their research in the sciences, including neuroscience? They should apply! Deadline is Nov 1. #uchicago #ai #neuro rb.gy/hwxbb
-
Ari Holtzman
@universeinanegg
love to see one of the most articulate ML researchers I know describe some of the key epistemological issues of the Large Model Era
Naomi Saphra
@nsaphra
New manifesto! How can AI scientists borrow from evolutionary biology to strengthen their evidence and claims? Featuring the beautiful eggs of the tawny-flanked prinia. nsaphra.net/post/prinia/
-
Ari Holtzman reposted
TTIC
@TTIC_Connect
Monday, September 18 at 11:30am: Talks at TTIC presents @Kanishkamisra of @UTCompSci with a talk titled "Analyzing Robust Conceptual Knowledge and Property Inheritance in Language Models." Please join us in Room 530 or via Panopto: uchicago.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=…
-
Ari Holtzman
@universeinanegg
The fact that I have to meticulously check my spam folder every day probably means it shouldn’t exist.
-
Ari Holtzman reposted
Zhijing Jin
@ZhijingJin
Excited to announce that I'm on the #nlproc job market this year! Actively seeking assistant professorship positions with my research on Causal NLP and Responsible AI. Looking forward to new opportunities and collaborations! ➡️zhijing-jin.com
#jobsearch #academictwitter -
Ari Holtzman
@universeinanegg
It’s time to start teasing out what human-like media generated by AI systems means to its viewers, not just whether it’s human-like! But how do we escape getting trapped in Turing-test like comparisons?
We wrote a little piece about some pitfalls of interpreting generated media.
Jessica Hullman
@JessicaHullman
Now for something a little different - A new essay on our tendencies to enact a kind of aestheticized Turing test w/ generative AI, one that inherits the type of judgments we apply to art 🎨
W/ Ari Holtzman @universeinanegg Andrew Gelman @statmodeling
statmodeling.stat.columbia.edu/2023/08/16/artificial-inte… -
Ari Holtzman reposted
Tim Dettmers
@Tim_Dettmers
Catch my talk on k-bit Inference Scaling Laws at the @ESFoMo workshop, ballroom A (fourth floor), 10:50am.
Slides: docs.google.com/presentation/d/1UEvCXnn8aWLQx5JS17zThojku… -
Ari Holtzman
@universeinanegg
Jesse is the person I trust most to tell me "actually interesting goals" when it comes to situated language use in the real, physical world.
Check this out.
Jesse Thomason
@_jessethomason_
If you're at ICML, head to 315 if you want to see me tell you what I think are actually interesting goals for RoboNLP in the age of All of This Going On Now; talk starting in a few minutes.
-
Ari Holtzman
@universeinanegg
We need a lot more discussion about what explanations should be doing and what users should be *doing* with explanations in the first place!
Chenhao's talk breaks down this problem space elegantly and shows how we might begin to approach it
Chenhao Tan
@ChenhaoTan
Unfortunately, I cannot make #icml in person, but check out my pre-recorded talk for the AI&HCI workshop: www.youtube.com/watch?v=QlOuWbPECqM, titled "Three Lessons Towards Human-Centered Explanations"!
-
Ari Holtzman
@universeinanegg
I’m around today and talking to folks about what the future of studying generative models might look like, how to meaningfully decompose language model behavior into useful categories, and how to do science as generalization becomes difficult to assess!
If you see me say hello!
Ari Holtzman
@universeinanegg
While demand for generative model training soars 📈, I think a new field is coalescing that’s focused on trying to make sense of generative models _once they’re already trained_: characterizing their behaviors, differences, and underlying mechanisms…so we wrote a paper about it!
-
Ari Holtzman
@universeinanegg
Hinton’s talk reminded me that powerpoints are dominated by provocative examples that feel important but systems are justified by ways you can use them.
-
Ari Holtzman
@universeinanegg
This probably speaks to the need for NLP to think hard about what questions we want to try to answer!
I'm not at all convinced that we're running out of questions to answer precisely, so much as doing pretty well at most tasks we come-up with is now much easier with LLMs.
Stephen Mayhew
@mayhewsw
It's the first ACL post-ChatGPT. What will the conversations/questions look like? Fill out this bingo card for a chance to win! #acl2023nlp
-
Ari Holtzman
@universeinanegg
Anyone know how to connect to the wifi at ACL?
-
Ari Holtzman
@universeinanegg
In other news, I’ll be joining @UChicagoCS and @DSI_UChicago in 2024 as an assistant professor and doing a postdoc @Meta in the meantime!
I’m at ACL in person and recruiting students who want to find fresh approaches to working with generative models so if that’s you let’s chat!
Ari Holtzman
@universeinanegg
While demand for generative model training soars 📈, I think a new field is coalescing that’s focused on trying to make sense of generative models _once they’re already trained_: characterizing their behaviors, differences, and underlying mechanisms…so we wrote a paper about it!
-
Ari Holtzman
@universeinanegg
While demand for generative model training soars 📈, I think a new field is coalescing that’s focused on trying to make sense of generative models _once they’re already trained_: characterizing their behaviors, differences, and underlying mechanisms…so we wrote a paper about it!
Post not available here, view on X
Ari Holtzman
@universeinanegg
We think that studying the *behavior* of generative models—predictable patterns in their outputs that we can reliably model like copying from recent tokens—can serve as a layer between benchmark results and model architecture to help us explain what makes generative models work.
Post not available here, view on X
Ari Holtzman
@universeinanegg
This all happened thanks to my simply amazing co-authors @PeterWestTM and @LukeZettlemoyer! We had a series of captivating conversations around these issues that led to this work and I feel like my viewpoints have both changed and clarified quite a bit through the process.
Ari Holtzman
@universeinanegg
Last but not least, thank you to all folks who we had fantastic discussions with and who gave super insightful feedback! This paper would not be half of what it is without them:
@dallascard
@jaredlcm
@dan_fried
@gabriel_ilharco
@Tim_Dettmers
@alisawuffles
@IanMagnusson
@alephic2 -
Ari Holtzman
@universeinanegg
simple ideas are cool and good :)
Joey Hejna
@JoeyHejna
Our work from AAAI 2023 is now on arxiv (very overdue)! The idea is really simple -- predicting language instructions during training improves generalization at test-time, particularly for combinatorial tasks.
-
Ari Holtzman
@universeinanegg
new genre just dropped
-
Ari Holtzman
@universeinanegg
I really want to write a paper called "Survivorship Biases in NLP Benchmarking" but if I believe my own thesis it probably wouldn't be of much use...
-
Ari Holtzman reposted
Sewon Min
@sewon__min
Check out our new work that tries to make the evaluation of LM's factuality📘 easier & simpler🚗 w/o compromising thoroughness🔎
Kalpesh Krishna
@kalpeshk2011
Factuality in long-form generation is hard to evaluate because (1) we don't know how to assign an accuracy value when a generation has mixed pieces of true/false info, and (2) human evaluation is extremely costly.
But from now on, you can use FActScore!
martiansideofthemoon.github.io/assets/factscore.pdf -
Ari Holtzman reposted
younes
@yb2698
A huge day for open source! 🔥
You can now load models from @huggingface in 4bit precision using load_in_4bit and bitsandbytes library, with no performance degradation.
Announcement notes here: huggingface.co/blog/4bit-transformers-bitsandbytes
Useful resources below
Tim Dettmers
@Tim_Dettmers
QLoRA: 4-bit finetuning of LLMs is here! With it comes Guanaco, a chatbot on a single GPU, achieving 99% ChatGPT performance on the Vicuna benchmark:
Paper: arxiv.org/abs/2305.14314
Code+Demo: github.com/artidoro/qlora
Samples: colab.research.google.com/drive/1kK6xasHiav9nhiRUJjPMZb4f…
Colab: colab.research.google.com/drive/17XEqL1JcmVWjHkT-WczdYkJl… -
Ari Holtzman reposted
Tim Dettmers
@Tim_Dettmers
QLoRA: 4-bit finetuning of LLMs is here! With it comes Guanaco, a chatbot on a single GPU, achieving 99% ChatGPT performance on the Vicuna benchmark:
Paper: arxiv.org/abs/2305.14314
Code+Demo: github.com/artidoro/qlora
Samples: colab.research.google.com/drive/1kK6xasHiav9nhiRUJjPMZb4f…
Colab: colab.research.google.com/drive/17XEqL1JcmVWjHkT-WczdYkJl… -
Ari Holtzman reposted
Mike Lewis
@ml_perception
New paper on scaling language models to sequences of a million bytes! MegaByte splits long byte sequences into fixed-size patches (analogous to tokens), then runs a large model between the patches, and a small model to predict each patch byte-by-byte. 1/
AK
@_akhaliq
MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers
abs: arxiv.org/abs/2305.07185
paper page: huggingface.co/papers/2305.07185 -
Ari Holtzman
@universeinanegg
finetuning 65B parameter models on a single GPU!
we're lowering the barrier to experimentation, which is one of the things I care about most in science!
Tim Dettmers
@Tim_Dettmers
The 4-bit bitsandbytes private beta is here! Our method, QLoRA, is integrated with the HF stack and supports all models. You can finetune a 65B model on a single 48 GB GPU. This beta will help us catch bugs and issues before our full release. Sign up:
forms.gle/QCxrUmXJ4RCbrkUL6 -
Ari Holtzman reposted
Gabriel Ilharco
@gabriel_ilharco
Today we are releasing a CLIP ViT-L/14 model with 79.2% zero-shot accuracy on ImageNet.
Our model outperforms OpenAI's CLIP by a large margin, and outperforms even bigger models (ViT-g/14) trained on LAION-2B
Check it out at huggingface.co/laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K! -
Ari Holtzman reposted
Victor Veitch 🔸
@victorveitch
Got something cool cooking in spuriousness, robust learning, or the use of invariance? Send it in!
Claudia Shi
@causalclaudia
Thrilled to share that the "Spurious Correlations, Invariance, and Stability" workshop is returning to @ICML2023 this year!
Submit your work (by May 23) on all things spurious correlations! See full CFP in
sites.google.com/view/scis-workshop-23 -
Ari Holtzman
@universeinanegg
Are there any easy to use open-source systems for using different language models with a mix of the ChatGPT UI (e.g. dialogue-like interface, session management, etc.) and the OpenAI playground (e.g. alterable parameters, alternative token probabilities, etc.)?
-
Ari Holtzman reposted
Tim Dettmers
@Tim_Dettmers
We now have Int8 backprop support for all GPUs for bitsandbytes! Now available via 'pip install bitsandbytes'. This was a contribution from @sasha_borzunov. We will release Int8 fine-tuning for all @huggingface models soon — stay tuned!
github.com/TimDettmers/bitsandbytes/releases/tag/0.37.0 -
Ari Holtzman reposted
Kalpesh Krishna
@kalpeshk2011
Human feedback plays a critical role in evaluating and aligning LLMs. However, evaluating long generated text is challenging, high-variance & expensive.
We present the LongEval guidelines to improve human evaluation of faithfulness in long summaries.
arxiv.org/abs/2301.13298
1/👇 -
Ari Holtzman reposted
Zhijing Jin
@ZhijingJin
Bravo🎉 to @flavioschneider's work: "Moûsai: Text-to-Music Generation" at @eth_en @mpi_is w/ us @bschoelkopf @ZhijingJin. We introduce a diffusion-based music🎼/audio generative model & GitHub w/ 800+ stars🔥
📔 : arxiv.org/abs/2301.11757
🎵 : flavioschneider.notion.site/Audio-Generation-with-Diffusi… @harmonai_org -
Ari Holtzman reposted
Armen Aghajanyan
@ArmenAgha
I have worked as a research scientist at FAIR for the last couple of years, and I have no degree. End of the day, getting a Ph.D. in ML is dependent on your goals.
Machine learning is a field that moves incredibly fast, and only a few graduate programs are capable of keeping up.
Behnam Neyshabur
@bneyshabur
These days, many people are interested in getting a PhD in ML. I think you should think really hard before committing to a PhD program in ML. Why?
I'm going to summarize some thoughts in this thread:
1/10 -
Ari Holtzman reposted
Tim Dettmers
@Tim_Dettmers
We ran +35,000 zero-shot experiments for our work on k-bit Inference Scaling Laws📈. A 30B 8-bit and 60B 4-bit LLM have the same model bits/inference latency, but different zero-shot accuracy. What is the best trade-off? The answer is clear: 4-bit is best
arxiv.org/abs/2212.09720 -
Ari Holtzman reposted
Dallas Card
@dallascard
The University of Michigan is part of a massive new program to support postdocs working on AI for Science. Applications are open for the 2023 cohort (due January 15th). If you are excited about the possibilities of AI for science, feel free to reach out!
news.umich.edu/u-m-schmidt-futures-to-partner-on-new-ai-r… -
Ari Holtzman reposted
Gabriel Ilharco
@gabriel_ilharco
Introducing task vectors!
A new way to steer models by doing arithmetic with model weights. Subtract to make models forget, add to make them learn
📜: arxiv.org/abs/2212.04089
🖥️: github.com/mlfoundations/task_vectors -
Ari Holtzman reposted
Tim Dettmers
@Tim_Dettmers
Catch my keynote on 8-bit Methods for Efficient Deep Learning, today at 4:35pm, ballroom C. Besides my work on 8-bit, I will also give a sneak peek into my latest project: Bit-level scaling laws for zeroshot inference. An analysis of 35,000 zeroshot experiments #neurips
-
Ari Holtzman
@universeinanegg
I think games that involve natural language communication, with their realizable outcomes and pragmatic reasoning, are the next big testbed for NLP models
Mike Lewis
@ml_perception
New paper in Science today on playing the classic negotiation game "Diplomacy" at a human level, by connecting language models with strategic reasoning! Our agent engages in intense and lengthy dialogues to persuade other players to follow its plans. This was really hard! 1/5
-
Ari Holtzman reposted
Xiang Lisa Li
@XiangLisaLi2
arxiv.org/abs/2210.15097
We propose contrastive decoding (CD), a more reliable search objective for text generation by contrasting LMs of different sizes. CD takes a large LM (expert LM e.g. OPT-13b) and a small LM (amateur LM e.g. OPT-125m) and maximizes their logprob difference -
Ari Holtzman reposted
Kaj Bostrom
@alephic2
Just released 🛠️PromptFab, a tiny library I've been using to make in-context learning experiments easier.
It helps you write reusable templates that surface your example format in any way you want, no boilerplate needed.
github.com/alephic/prompt-fab -
Ari Holtzman reposted
arXiv Daily
@Arxiv_Daily
What Do NLP Researchers Believe? Results of the NLP Community Metasurvey
deepai.org/publication/what-do-nlp-researchers-believe-re…
by Julian Michael et al. including @universeinanegg
#computation #language -
Ari Holtzman
@universeinanegg
the NLP Community Metasurvey results are released!
Julian Michael
@_julianmichael_
🚨The wait is over! Results are out for the NLP Community Metasurvey!🚨
What do NLPers think NLPers think? 🧐💭(🧐💭)❓
There's a ton here, so we recommend anyone interested in a deeper dive tab away from Twitter and read the full report: nlpsurvey.net/nlp-metasurvey-results.pdf
Brief 🧵: -
Ari Holtzman
@universeinanegg
What do these outlier features that emerge in large language models represent?
Tim Dettmers
@Tim_Dettmers
We release LLM.int8(), the first 8-bit inference method that saves 2x memory and does not degrade performance for 175B models by exploiting emergent properties. Read More:
Paper: arxiv.org/abs/2208.07339
Software: huggingface.co/blog/hf-bitsandbytes-integration
Emergence: timdettmers.com/2022/08/17/llm-int8-and-emergent-features/ -
Ari Holtzman
@universeinanegg
First steps in post-hoc modular revision of large models
Gabriel Ilharco
@gabriel_ilharco
The year is 2032. A model was trained on all images, videos and text on the web, using over 100 yottaFLOPs.
It still thinks this is an image of a dog.
To fix models post-hoc, check out PAINT!🎨
📜 arxiv.org/abs/2208.05592
💻 github.com/mlfoundations/patching
🌐 model-patching.github.io -
Ari Holtzman
@universeinanegg
I think the most important thing for figuring out the right hypotheses to explain LM behavior is just giving people access to these models, so they can poke at them on their own, for free.
Now you can run a 13B model in colab or a 176B model on a single node.
What will we find?
Tim Dettmers
@Tim_Dettmers
We release the public beta for bnb-int8🟪 for all @huggingface 🤗models, which allows for Int8 inference without performance degradation up to scales of 176B params 📈. You can run OPT-175B/BLOOM-176B easily on a single machine 🖥️. You can try it here: docs.google.com/document/d/1JxSo4lQgMDBdnd19VBEoaG-mMfQup…
1/n -
Ari Holtzman reposted
Zhijing Jin
@ZhijingJin
Just arrived in the Netherlands for #uai2022. Excited to talk to & learn from causality people here! 💡Keywords I'm interested in: causality, robustness, epistemology, language models. I'll also have a poster at the Causal Representation Learning workshop @crl_uai. See you there!
-
Ari Holtzman
@universeinanegg
prediction: as ML automates more parts of the process, the book->movie pipeline will be in competition with the book->video game pipeline, in which counterfactual choices can be explored, which is a lot of what people want out of fan fiction (the most popular genre of fiction)
-
Ari Holtzman
@universeinanegg
starting to think "compositionality" might be NLP's "consciousness"
-
Ari Holtzman
@universeinanegg
How soon until Large Language Model Syndrome (LLMS) is a Wikipedia article?
depths of wikipedia!
@depthsofwiki
-
Ari Holtzman reposted
Daniel Fried
@dan_fried
If you're planning to pretrain a large generative language model, consider using an infilling objective:
Mo Bavarian
@mobav0
1/ Introducing FIM our comprehensive study of infilling in decoder-only LMs at @OpenAI
📄 arxiv.org/abs/2207.14255
Main lessons:
1/ FIM models are strictly superior to regular left-to-right models. They achieve
the same left-to-right evals while learning to infill -
Ari Holtzman
@universeinanegg
wow, this is exactly the kind of work I want to see more of: trying to reason about what causal variables lead to LM behavior and developing methods to do this from purely observational data because retraining a million huge models is just infeasible!
Yanai Elazar
@yanaiela
New *very exciting* paper
We causally estimate the effect of simple data statistics (e.g. co-occurrences) on model predictions.
Joint work w/ @KassnerNora, @ravfogel, @amir_feder, @Lasha1608, @mariusmosbach, @boknilev, Hinrich Schütze, and @yoavgo
arxiv.org/abs/2207.14251 -
Ari Holtzman
@universeinanegg
Does anyone if the *SEM 2022 talks were recorded at NAACL?
Been referring to them enough that I really ought to just have people watch them at this point...
@_starsem? -
Ari Holtzman reposted
Daniel Fried
@dan_fried
This is today! 8:30am-5pm Pacific. Excited for our papers, speakers, and breakout discussions!
Schedule: unimplicit2022.github.io
Room: 701 Clallum at NAACL
Virtual: underline.io/events/325/sessions?eventSessionId=11206
#naacl2022
Valentina Pyatkin
@valentina__py
We are looking forward to an exciting line-up of invited speakers at the UnImplicit workshop #naacl2022
unimplicit2022.github.io
@complingy @meanwhileina @dan_fried @TalitaAnthonio @microth @rtsarfaty @yoavgo -
Ari Holtzman
@universeinanegg
me at NAACL
What a week, huh? all Wednesdays
@whataweekhuh
-
Ari Holtzman reposted
Peter West
@PeterWestTM
Very excited to present Symbolic Knowledge Distillation at NAACL tomorrow (July 13)! For those early risers, I will be speaking at the ML for NLP session (8:00 - 9:00 a.m) at 8:30 in Quinault. For those who sleep in, come find me for a chat around the conference!
Peter West
@PeterWestTM
I'm excited to introduce Symbolic Knowledge Distillation!
⭐️Commonsense knowledge with a twist: generated by models 🤖 filtered by models 🤖 & learned by models 🤖
⭐️Human-quality knowledge, with (almost) no humans 🙅
⭐️A student model 🤖 that beats its teacher 🧑🏫 and more!
(1/2) -
Ari Holtzman
@universeinanegg
I’m at NAACL and excited to discuss the shift in NLP towards discovering, explaining, and using unplanned behavior in models!
Exploiting differences in artificial vs. biological neural networks that make discovering behavior easier in the former seems promising.
Some ideas:
Ari Holtzman
@universeinanegg
I'm noticing a shift in NLP from designing models for tasks to discovering novel behavior in models *after training*, e.g. in-context learning. But I think we're missing key vocabulary to breakdown model behavior, as if we were trying to explain steam without the concept of water
Post not available here, view on X
Ari Holtzman
@universeinanegg
4. Ease of Perturbation—unlike studying the human brain, we can delete a single neuron in our models or simultaneously perturb every single one in the exact same way and see what happens, leading to a massive amount of possible interventions for discovering structure.
Ari Holtzman
@universeinanegg
5. No Observer Effects—a big issue with studying living things is that simply by being present or putting them in a laboratory, you often change their behavior. Language models don’t distinguish between training and inference time, unless we force them to (e.g., dropout).
-
Ari Holtzman reposted
Daniel Fried
@dan_fried
This Friday at NAACL! Hope to see many folks there, and throughout the conference.
Valentina Pyatkin
@valentina__py
We are looking forward to an exciting line-up of invited speakers at the UnImplicit workshop #naacl2022
unimplicit2022.github.io
@complingy @meanwhileina @dan_fried @TalitaAnthonio @microth @rtsarfaty @yoavgo -
Ari Holtzman
@universeinanegg
both of these look really cool…
Greg Durrett
@gregd_nlp
Two papers from my group at #naacl2022:
@JiachengNLP, Sid J Reddy: lattice decoding for text generation
Sess. 1B talk; arxiv.org/pdf/2112.07660.pdf
@yasumasa_onoe, @mjqzhang, @eunsolc: a cloze task for testing entity knowledge in language models
9F poster; arxiv.org/pdf/2205.02832.pdf -
Ari Holtzman
@universeinanegg
I'm noticing a shift in NLP from designing models for tasks to discovering novel behavior in models *after training*, e.g. in-context learning. But I think we're missing key vocabulary to breakdown model behavior, as if we were trying to explain steam without the concept of water
Ari Holtzman
@universeinanegg
If you'll be at NAACL and are interested in this, let's talk! It feels especially important now: most researchers can't afford to pretrain the large models that dominate leaderboards..so let's figure out more ways to do good behavioral analysis without training them from scratch!
Ari Holtzman
@universeinanegg
Also, I might as well mention that I'm planning to apply to faculty jobs in the autumn (mostly in North America, but that's negotiable). More on that soon!
-
Ari Holtzman
@universeinanegg
it finally happened...I got the "are you a robot?" test from searching google scholar too much...
-
Ari Holtzman
@universeinanegg
PSA: I find lexico.com super helpful for writing papers when you need a synonym that fits *just* right
-
Ari Holtzman
@universeinanegg
There's less than a week left to help us figure out what opinions NLPers hold and what opinions they believe they're "responding to" i.e. what opinion NLPers think other NLPers hold.
This would be such a useful reference point for the field, so please fill out our survey!
Julian Michael
@_julianmichael_
📣 CALLING ALL NLP RESEARCHERS 📣
Do you think you have lots of minority opinions on NLP issues? Where does the research community stand? Where does the community *think* it stands? Let’s find out — take part in the NLP Community Metasurvey!
🧠💭🧠 nlpsurvey.net -
Ari Holtzman
@universeinanegg
I'm really surprised no one has trained a big backwards LM, yet.
There are loads of uses:
- ensembling possible futures as a meaning representation
- ranking generations/answers by their PMI with the prompt
- planning around how well partial generations predict meta-data
etc.
Christian Szegedy
@ChrSzegedy
I am quite convinced that if you trained a reverse GPT model, that predicts in the backward direction, then going backward from the correct answer, one could automatically produce the best prefixes.
Feel free to cite this tweet, if you managed to make it work. :)
Ari Holtzman
@universeinanegg
The first was done by West et al. in arxiv.org/abs/2010.08566 albeit with a smaller LM than we would have liked.
The second was done by us in arxiv.org/abs/2104.08315 but we had to do it a round-about way, we would have preferred a backwards LM.
The third one is up for grabs...
























































































































































































































































































































