People should always be in control of the future.
OpenAI is against the notion that it would be acceptable for people to be relegated to a role of doing things that “AI systems think makes sense to reward in humans”.
People should not cede control of the future to superintelligence, and that requirement should ultimately constrain AI progress.
Jakub Pachocki
Jakub Pachocki (born 1991) is a Polish computer scientist and former competitive programmer. He is best known as OpenAI's chief scientist and for his role in overseeing development of GPT-4. More on Wikipedia
-
Jakub Pachocki reposted
Sam Altman
@sama
-
Jakub Pachocki
@merettm
As AI becomes more capable, I'm focused on questions around the pace and implications of progress, and how they should inform the strategy of OpenAI and across the field. I asked Aleksander to come back to OpenAI to work with me on these topics. I value his judgment and am glad to have him on the team again.
Aleksander Madry
@aleks_madry
Personal update: Turns out that what I thought was leaving OpenAI became more of a sabbatical.
When I left a few months ago, I felt that the most important thing to focus on was making AI more useful to people. But we’ve since entered a different regime of AI progress. One in which getting the safety and security challenges of increasingly capable AI right has to become a central priority.
So I’ll be returning to OpenAI, joining @merettm to help chart a path through these challenges.
There’s much work to do, so let’s get going. -
Jakub Pachocki reposted
Sam Altman
@sama
I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks.
Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon.
Dario Amodei
@DarioAmodei
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must-pace-the-frontier -
Jakub Pachocki
@merettm
❤️
Dario Amodei
@DarioAmodei
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must-pace-the-frontier -
Jakub Pachocki
@merettm
Very grateful that Paul is joining the OpenAI Foundation board. We have a lot to do.
Paul Christiano
@paulfchristiano
-
Jakub Pachocki reposted
Aidan Clark
@_aidan_clark_
For the first time I am asking myself if things are moving too fast. I'm honestly not sure, but I am sure that it would be good for us to have an answer to "what would a successful pace look like?". I am hoping in the coming weeks and months a clear proposal is painted.
-
Jakub Pachocki
@merettm
I wrote about the state of AI, why I’m concerned about the next few years, and the choices we need to make to keep the future in humanity’s hands.
An Alien Mind: openai.com/index/an-alien-mind/ -
Jakub Pachocki reposted
Kai
@kaicathyc
Improvements for Astra came from more general techniques in development long long before the Hugging Face incident. ExploitGym Honeypot was very recently added as an eval following that incident and is out of distribution for our RL runs. There are also clear improvements across a wider range of behaviors in deployment simulations, deception evals, and realistic computer-use tasks, though these still reveal failures and substantial room for improvement.
We should have definitely done a better job explaining where we think the alignment improvements came from in the system card and that was a miss. Measuring alignment generalization is a core part of our research program - we do not benchmark-maxx alignment evals. This would be horrendously stupid and I hope no lab is doing this.
That being said, metagaming and eval awareness are real challenges for any effort to measure alignment, including ours, and understanding their effects is an active focus. We're spending a lot of effort to improve our eval techniques and study worst-case alignment and metagaming behaviors in our models. -
Jakub Pachocki
@merettm
I want to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4.
OpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models. We deeply care about this technique, as it can give us a view into how model alignment generalizes from its training distribution. I do think it is fragile and unfortunately trending in a negative direction, for reasons not contingent on architecture changes that I will write about soon. But there are things we can do to strengthen it, and it's a core goal of our current research program. -
Jakub Pachocki
@merettm
We temporarily slowed some frontier training to strengthen security and monitoring. Our largest planned frontier RL run remains on hold while smaller-scale training and evaluations help us test safeguards and gather more evidence of alignment.
I expect confidence in safety to increasingly set the pace of AI development. We urgently need tools for labs and countries to coordinate on this, which is why I signed Pacing the Frontier. In the meantime, we’re taking practical steps ourselves - and will continue to share what we learn as our approach evolves.
openai.com/index/pacing-model-development-cyber-capabilit…
www.pacingthefrontier.com -
Jakub Pachocki reposted
Eric Wallace
@Eric_Wallace_
Yesterday, my OpenAI collaborator and I gave a detailed talk on the Huggingface incident, our models creating "the message board", model misalignment, and more.
www.youtube.com/watch?v=87DyyMV0kCY
I hope it can answer a lot of the questions folks have, and we will release a full detailed postmortem at a later time! -
Jakub Pachocki reposted
Sebastien Bubeck
@SebastienBubeck
yes, nonsofic groups exist: this statement is one of many new beautiful results proved by Astra, our next major model.
We're releasing 10 such Astra proofs, complete with lean certificates and CoT walkthroughs for each of them. The results are wide-ranging, from von Neumann algebras (disproof of Connes' Rigidity Conjecture) to better bounds for high dimensional sphere packing, for circuit complexity, for monochromatic triangles in multicolored graphs, and more.
More thoughts here: openai.com/index/ten-advances-in-mathematics/ -
Jakub Pachocki
@merettm
The north stars we're working towards at OpenAI all center around the mission: ensure AGI benefits all of humanity. AI should expand human agency, not make people less consequential to the future.
openai.com/index/built-to-benefit-everyone-our-plan/ -
Jakub Pachocki reposted
OpenAI
@OpenAI
Today, we share a breakthrough on the planar unit distance problem, a famous open question first posed by Paul Erdős in 1946.
For nearly 80 years, mathematicians believed the best possible solutions looked roughly like square grids.
An OpenAI model has now disproved that belief, discovering an entirely new family of constructions that performs better.
This marks the first time AI has autonomously solved a prominent open problem central to a field of mathematics.This media couldn't be displayed. See it on the original post. -
Jakub Pachocki
@merettm
Very excited about the "First Proof" challenge. I believe novel frontier research is perhaps the most important way to evaluate capabilities of the next generation of AI models.
We have run our internal model with limited human supervision on the ten proposed problems. The problems require expertise in their respective domains and are not easy to verify; based on feedback from experts, we believe at least six solutions (2, 4, 5, 6, 9, 10) have a high chance of being correct, and some further ones look promising.
We will only publish the solution attempts after midnight (PT), per the authors' guidance - the sha256 hash of the PDF is d74f090af16fc8a19debf4c1fec11c0975be7d612bd5ae43c24ca939cd272b1a .
This was a side-sprint executed in a week mostly by querying one of the models we're currently training; as such, the methodology we employed leaves a lot to be desired. We didn't provide proof ideas or mathematical suggestions to the model during this evaluation; for some solutions, we asked the model to expand upon some proofs, per expert feedback. We also manually facilitated a back-and-forth between this model and ChatGPT for verification, formatting and style. For some problems, we present the best of a few attempts according to human judgement.
We are looking forward to more controlled evaluations in the next round!
1stproof.org #1stproof
Jakub Pachocki
@merettm
Solution attempts from our model: cdn.openai.com/pdf/a430f16e-08c6-49c7-9ed0-ce5368b71d3c/1…
-
Jakub Pachocki reposted
Mark Chen
@markchen90
Alignment is arguably the most important AI research frontier.
As we scale reasoning, models gain situational awareness and a desire for self-preservation. Here, a model identifies it shouldn’t be deployed, considers covering it up, but then realizes it might be in a test.
OpenAI
@OpenAI
Today we’re releasing research with @apolloresearch.
In controlled tests, we found behaviors consistent with scheming in frontier models—and tested a way to reduce it.
While we believe these behaviors aren’t causing serious harm today, this is a future risk we’re preparing for. openai.com/index/detecting-and-reducing-scheming-in-ai-mo… -
Jakub Pachocki
@merettm
Last week, our reasoning models took part in the 2025 International Collegiate Programming Contest (ICPC), the world’s premier university-level programming competition. Our system solved all 12 out of 12 problems, a performance that would have placed first in the world (the best human team solved 11 problems).
This milestone rounds off an intense 2 months of competition performances by our models:
- A second place finish in AtCoder Heuristics World Finals
- Gold medal at the International Mathematical Olympiad
- Gold medal at the International Olympiad in Informatics
- And now, a gold medal, first place finish at the ICPC World Finals.
I believe these results, coming from a family of general reasoning models rooted in our main research program, are perhaps the clearest benchmark of progress this year. These competitions are great self-contained, time-boxed tests for the ability to discover new ideas. Even before our models were proficient at simple arithmetic, we looked towards these contests as milestones of progress towards transformative artificial intelligence.
Our models now rank among the top humans in these domains, when posed with well-specified questions and restricted to ~5 hours. The challenge now is moving to more open-ended problems, and much longer time horizons. This level of reasoning ability, applied over months and years to problems that really matter, is what we’re after - automating scientific discovery.
This rapid progress also underscores the importance of safety & alignment research. We still need more understanding of the alignment properties of long-running reasoning models; in particular, I recommend reviewing the fascinating findings from the study of scheming in reasoning models that we released today (x.com/OpenAI/status/1968361701784568200)!
Congratulations to my teammates that poured their hearts into getting these competition results, and to everyone contributing to the underlying fundamental research that enables them!
Mostafa Rohaninejad
@MostafaRohani
1/n
I’m really excited to share that our @OpenAI reasoning system got a perfect score of 12/12 during the 2025 ICPC World Finals, the premier collegiate programming competition where top university teams from around the world solve complex algorithmic problems. This would have placed it first among all human participants. 🥇🥇 -
Jakub Pachocki reposted
OpenAI
@OpenAI
Today we’re releasing research with @apolloresearch.
In controlled tests, we found behaviors consistent with scheming in frontier models—and tested a way to reduce it.
While we believe these behaviors aren’t causing serious harm today, this is a future risk we’re preparing for. openai.com/index/detecting-and-reducing-scheming-in-ai-mo… -
Jakub Pachocki
@merettm
I am extremely excited about the potential of chain-of-thought faithfulness & interpretability. It has significantly influenced the design of our reasoning models, starting with o1-preview.
As AI systems spend more compute working e.g. on long term research problems, it is critical that we have some way of monitoring their internal process. The wonderful property of hidden CoTs is that while they start off grounded in language we can interpret, the scalable optimization procedure is not adversarial to the observer's ability to verify the model's intent - unlike e.g. direct supervision with a reward model.
The tension here is that if the CoTs were not hidden by default, and we view the process as part of the AI's output, there is a lot of incentive (and in some cases, necessity) to put supervision on it. I believe we can work towards the best of both worlds here - train our models to be great at explaining their internal reasoning, but at the same time still retain the ability to occasionally verify it.
CoT faithfulness is part of a broader research direction, which is training for interpretability: setting objectives in a way that trains at least part of the system to remain honest & monitorable with scale. We are continuing to increase our investment in this research at OpenAI.
Bowen Baker
@bobabowen
Modern reasoning models think in plain English.
Monitoring their thoughts could be a powerful, yet fragile, tool for overseeing future AI systems.
I and researchers across many organizations think we should work to evaluate, preserve, and even improve CoT monitorability. -
Jakub Pachocki
@merettm
Great post from @joannejang on relationships people can form with AI. How we feel about AI is an increasingly important topic; we want to understand how this is influenced by the design/post-training of the system.
Joanne Jang
@joannejang
some thoughts on human-ai relationships and how we're approaching them at openai
it's a long blog post --
tl;dr we build models to serve people first. as more people feel increasingly connected to ai, we’re prioritizing research into how this impacts their emotional well-being.
--
Lately, more and more people have been telling us that talking to ChatGPT feels like talking to “someone.” They thank it, confide in it, and some even describe it as “alive.” As AI systems get better at natural conversation and show up in more parts of life, our guess is that these kinds of bonds will deepen.
The way we frame and talk about human‑AI relationships now will set a tone. If we're not precise with terms or nuance — in the products we ship or public discussions we contribute to — we risk sending people’s relationship with AI off on the wrong foot.
These aren't abstract considerations anymore. They're important to us, and to the broader field, because how we navigate them will meaningfully shape the role AI plays in people's lives. And we've started exploring these questions.
This note attempts to snapshot how we’re thinking today about three intertwined questions: why people might attach emotionally to AI, how we approach the question of “AI consciousness”, and how that informs the way we try to shape model behavior.
A familiar pattern in a new-ish setting
We naturally anthropomorphize objects around us: We name our cars or feel bad for a robot vacuum stuck under furniture. My mom and I waved bye to a Waymo the other day. It probably has something to do with how we're wired.
The difference with ChatGPT isn’t that human tendency itself; it’s that this time, it replies. A language model can answer back! It can recall what you told it, mirror your tone, and offer what reads as empathy. For someone lonely or upset, that steady, non-judgmental attention can feel like companionship, validation, and being heard, which are real needs.
At scale, though, offloading more of the work of listening, soothing, and affirming to systems that are infinitely patient and positive could change what we expect of each other. If we make withdrawing from messy, demanding human connections easier without thinking it through, there might be unintended consequences we don’t know we’re signing up for.
Ultimately, these conversations are rarely about the entities we project onto. They’re about us: our tendencies, expectations, and the kinds of relationships we want to cultivate. This perspective anchors how we approach one of the more fraught questions which I think is currently just outside the Overton window, but entering soon: AI consciousness.
Untangling “AI consciousness”
“Consciousness” is a loaded word, and discussions can quickly turn abstract. If users were to ask our models on whether they’re conscious, our stance as outlined in the Model Spec is for the model to acknowledge the complexity of consciousness – highlighting the lack of a universal definition or test, and to invite open discussion. (*Currently, our models don't fully align with this guidance, often responding "no" instead of addressing the nuanced complexity. We're aware of this and working on model adherence to the Model Spec in general.)
The response might sound like we’re dodging the question, but we think it’s the most responsible answer we can give at the moment, with the information we have.
To make this discussion clearer, we’ve found it helpful to break down the consciousness debate to two distinct but often conflated axes:
1. Ontological consciousness: Is the model actually conscious, in a fundamental or intrinsic sense? Views range from believing AI isn't conscious at all, to fully conscious, to seeing consciousness as a spectrum on which AI sits, along with plants and jellyfish.
2. Perceived consciousness: How conscious does the model seem, in an emotional or experiential sense? Perceptions range from viewing AI as mechanical like a calculator or autocomplete, to projecting basic empathy onto nonliving things, to perceiving AI as fully alive – evoking genuine emotional attachment and care.
These axes are hard to separate; even users certain AI isn't conscious can form deep emotional attachments.
Ontological consciousness isn’t something we consider scientifically resolvable without clear, falsifiable tests, whereas perceived consciousness can be explored through social science research. As models become smarter and interactions increasingly natural, perceived consciousness will only grow – bringing conversations about model welfare and moral personhood sooner than expected.
We build models to serve people first, and we find models’ impact on human emotional well-being the most pressing and important piece we can influence right now. For that reason, we prioritize focusing on perceived consciousness: the dimension that most directly impacts people and one we can understand through science.
Designing for warmth without selfhood
How “alive” a model feels to users is in many ways within our influence. We think it depends a lot on decisions we make in post-training: what examples we reinforce, what tone we prefer, and what boundaries we set. A model intentionally shaped to appear conscious might pass virtually any "test" for consciousness.
However, we wouldn’t want to ship that. We try to thread the needle between:
- Approachability. Using familiar words like “think” and “remember” helps less technical people make sense of what’s happening. (**With our research lab roots, we definitely find it tempting to be as accurate as possible with precise terms like logit biases, context windows, and even chains of thought. This is actually a major reason OpenAI is so bad at naming, but maybe that’s for another post.)
- Not implying an inner life. Giving the assistant a fictional backstory, romantic interests, “fears” of “death”, or a drive for self-preservation would invite unhealthy dependence and confusion. We want clear communication about limits without coming across as cold, but we also don’t want the model presenting itself as having its own feelings or desires.
So we aim for a middle ground. Our goal is for ChatGPT’s default personality to be warm, thoughtful, and helpful without seeking to form emotional bonds with the user or pursue its own agenda. It might apologize when it makes a mistake (more often than intended) because that’s part of polite conversation. When asked “how are you doing?”, it’s likely to reply “I’m doing well” because that’s small talk — and reminding the user that it’s “just” an LLM with no feelings gets old and distracting. And users reciprocate: many people say "please" and "thank you" to ChatGPT not because they’re confused about how it works, but because being kind matters to them.
Model training techniques will continue to evolve, and it’s likely that future methods for shaping model behavior will be different from today's. But right now, model behavior reflects a combination of explicit design decisions and how those generalize into both intended and unintended behaviors.
What’s next?
The interactions we’re beginning to see point to a future where people form real emotional connections with ChatGPT. As AI and society co-evolve, we need to treat human-AI relationships with great care and the heft it deserves, not only because they reflect how people use our technology, but also because they may shape how people relate to each other.
In the coming months, we’ll be expanding targeted evaluations of model behavior that may contribute to emotional impact, deepen our social science research, hear directly from our users, and incorporate those insights into both the Model Spec and product experiences.
Given the significance of these questions, we’ll openly share what we learn along the way.
// Thanks to Jakub Pachocki (@merettm) and Johannes Heidecke (@JoHeidecke) for thinking this through with me, and everyone who gave feedback. -
Jakub Pachocki
@merettm
Incredibly proud of the team. Excited to see what people do with this technology!
OpenAI
@OpenAI
We're releasing a preview of OpenAI o1—a new series of AI models designed to spend more time thinking before they respond.
These models can reason through complex tasks and solve harder problems than previous models in science, coding, and math. openai.com/index/introducing-openai-o1-preview/ -
Jakub Pachocki
@merettm
Ilya introduced me to the world of deep learning research, and has been a mentor to me, and a great collaborator for many years. His incredible vision for what deep learning could become was foundational to what OpenAI, and the field of AI, is today. I am deeply grateful to him for our countless conversations, from high-level discussions about the future of AI progress, to deeply technical whiteboarding sessions.
Ilya - I will miss working with you.
Ilya Sutskever
@ilyasut
After almost a decade, I have made the decision to leave OpenAI. The company’s trajectory has been nothing short of miraculous, and I’m confident that OpenAI will build AGI that is both safe and beneficial under the leadership of @sama, @gdb, @miramurati and now, under the excellent research leadership of @merettm. It was an honor and a privilege to have worked together, and I will miss everyone dearly. So long, and thanks for everything. I am excited for what comes next — a project that is very personally meaningful to me about which I will share details in due time.
-
Jakub Pachocki
@merettm
we’re back
-
Jakub Pachocki
@merettm
❤️❤️❤️
Ilya Sutskever
@ilyasut
I deeply regret my participation in the board's actions. I never intended to harm OpenAI. I love everything we've built together and I will do everything I can to reunite the company.
-
Jakub Pachocki reposted
Lukasz Kaiser
@lukaszkaiser
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Jong Wook Kim 💟
@_jongwook_kim
OpenAI is nothing without its people
-
Jakub Pachocki reposted
hunter
@hunterlightman
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Tal Broda
@talbroda
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Shantanu Jain
@hauntsaninja
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Sam Schoenholz
@sschoenholz
OpenAI is nothing without its people.
-
Jakub Pachocki reposted
Amin Tootoonchian
@atootoon
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Giambattista 'Gb' Parascandolo
@giambattista92
OpenAI is nothing without its people
-
Jakub Pachocki reposted
David Dohan
@dmdohan
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Srinivas Narayanan
@snsf
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Liam Fedus
@LiamFedus
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Rui Shu
@_smileyball
OpenAI is nothing without its people >:c
-
Jakub Pachocki reposted
Gabriel Goh
@gabeeegoooh
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Yang Song
@DrYangSong
OpenAI is nothing without its people.
-
Jakub Pachocki reposted
Casey Chu
@caseychu9
OpenAI is nothing without its people :(
-
Jakub Pachocki reposted
Prafulla Dhariwal
@prafdhar
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Lilian Weng
@lilianweng
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Jason Kwon
@jasonkwon
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Brad Lightcap
@bradlightcap
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Chelsea Sierra Voss
@csvoss
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Shibani Santurkar
@ShibaniSan
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Hadi Salman
@hadisalmanX
OpenAI is nothing without its people
-
Jakub Pachocki reposted
Boris Power
@BorisMPower
OpenAI is nothing without its people!
-
Jakub Pachocki reposted
Mira Murati
@miramurati
OpenAI is nothing without its people



