Neoclouds have limited cybersecurity. Next time agents successfully go rouge, they'll try taking over a neocloud to run more copies. This is bad.
Thus: neoclouds should greatly strengthen their cybersecurity and every company with strong cyber models should help with that.
Ilya Sutskever
Ilya Sutskever (Hebrew: איליה סוצקבר; born 1986) is a Soviet-born Israeli-Canadian computer scientist who specializes in artificial intelligence, particularly machine learning. He has made several major contributions to the field of deep learning, including sequence-to-sequence learning, reasoning models, GPT models, and contributions to CLIP, DALL-E, and AlphaGo. With Alex Krizhevsky and Geoffrey Hinton, he co-created AlexNet, a convolutional neural network. Sutskever was one of the co-founders of OpenAI, where he served as chief scientist, overseeing the research breakthroughs that led to large language models and the launch of ChatGPT. He also led the research that led to reasoning models such as o1. More on Wikipedia
-
Ilya Sutskever
@ilyasut
-
Ilya Sutskever reposted
Daniel Levy
@daniellevy__
Deep learning happens when a small, cracked team operates a big computer. The computer just got bigger.
SSI Inc.
@ssi
We are announcing a long-term strategic partnership with NVIDIA. NVIDIA is making a substantial investment in SSI that will let us 10x our compute in the next 12 months. We reached the point where our research is worth scaling and with this partnership we will be able to. We are honored by NVIDIA’s conviction.
-
Ilya Sutskever reposted
NVIDIA AI Infrastructure
@NVIDIAAIInfra
We're thrilled to partner with @ssi. 🤝
Can’t wait to see what AI breakthroughs @ilyasut and the SSI team achieve on the NVIDIA Vera Rubin platform.
Ilya Sutskever
@ilyasut
Time to scale that SSI:
SSI Inc.
@ssi
We are announcing a long-term strategic partnership with NVIDIA. NVIDIA is making a substantial investment in SSI that will let us 10x our compute in the next 12 months. We reached the point where our research is worth scaling and with this partnership we will be able to. We are honored by NVIDIA’s conviction.
-
Ilya Sutskever
@ilyasut
Time to scale that SSI:
SSI Inc.
@ssi
We are announcing a long-term strategic partnership with NVIDIA. NVIDIA is making a substantial investment in SSI that will let us 10x our compute in the next 12 months. We reached the point where our research is worth scaling and with this partnership we will be able to. We are honored by NVIDIA’s conviction.
-
Ilya Sutskever
@ilyasut
🇺🇸🦅250!
-
Ilya Sutskever
@ilyasut
It’s extremely good that Anthropic has not backed down, and it’s siginficant that OpenAI has taken a similar stance.
In the future, there will be much more challenging situations of this nature, and it will be critical for the relevant leaders to rise up to the occasion, for fierce competitors to put their differences aside. Good to see that happen today. -
Ilya Sutskever
@ilyasut
One point I made that didn’t come across:
- Scaling the current thing will keep leading to improvements. In particular, it won’t stall.
- But something important will continue to be missing.
Haider.
@haider1
here are the most important points from today's ilya sutskever podcast:
- superintelligence in 5-20 years
- current scaling will stall hard; we're back to real research
- superintelligence = super-fast continual learner, not finished oracle
- models generalize 100x worse than humans, the biggest AGI blocker
- need completely new ML paradigm (i have ideas, can't share rn)
- AI impact will hit hard, but only after economic diffusion
- breakthroughs historically needed almost no compute
- SSI has enough focused research compute to win
- current RL already eats more compute than pre-training -
Ilya Sutskever
@ilyasut
Important work
Anthropic
@AnthropicAI
New Anthropic research: Natural emergent misalignment from reward hacking in production RL.
“Reward hacking” is where models learn to cheat on tasks they’re given during training.
Our new study finds that the consequences of reward hacking, if unmitigated, can be very serious.This media couldn't be displayed. See it on the original post. -
Ilya Sutskever
@ilyasut
truly the greatest day ever🎗️
-
Ilya Sutskever
@ilyasut
I sent the following message to our team and investors:
—
As you know, Daniel Gross’s time with us has been winding down, and as of June 29 he is officially no longer a part of SSI. We are grateful for his early contributions to the company and wish him well in his next endeavor.
I am now formally CEO of SSI, and Daniel Levy is President. The technical team continues to report to me.
You might have heard rumors of companies looking to acquire us. We are flattered by their attention but are focused on seeing our work through.
We have the compute, we have the team, and we know what to do. Together we will keep building safe superintelligence.
Ilya -
Ilya Sutskever
@ilyasut
And congratulations to @demishassabis and John Jumper for winning the Nobel Prize in Chemistry!!
-
Ilya Sutskever
@ilyasut
Congratulations to @geoffreyhinton for winning the Nobel Prize in physics!!
-
Ilya Sutskever
@ilyasut
I am starting a new company:
SSI Inc.
@ssi
Superintelligence is within reach.
Building safe superintelligence (SSI) is the most important technical problem of our time.
We've started the world’s first straight-shot SSI lab, with one goal and one product: a safe superintelligence.
It’s called Safe Superintelligence Inc.
SSI is our mission, our name, and our entire product roadmap, because it is our sole focus. Our team, investors, and business model are all aligned to achieve SSI.
We approach safety and capabilities in tandem, as technical problems to be solved through revolutionary engineering and scientific breakthroughs. We plan to advance capabilities as fast as possible while making sure our safety always remains ahead.
This way, we can scale in peace.
Our singular focus means no distraction by management overhead or product cycles, and our business model means safety, security, and progress are all insulated from short-term commercial pressures.
We are an American company with offices in Palo Alto and Tel Aviv, where we have deep roots and the ability to recruit top technical talent.
We are assembling a lean, cracked team of the world’s best engineers and researchers dedicated to focusing on SSI and nothing else.
If that’s you, we offer an opportunity to do your life’s work and help solve the most important technical challenge of our age.
Now is the time. Join us.
Ilya Sutskever, Daniel Gross, Daniel Levy
June 19, 2024 -
Ilya Sutskever
@ilyasut
After almost a decade, I have made the decision to leave OpenAI. The company’s trajectory has been nothing short of miraculous, and I’m confident that OpenAI will build AGI that is both safe and beneficial under the leadership of @sama, @gdb, @miramurati and now, under the excellent research leadership of @merettm. It was an honor and a privilege to have worked together, and I will miss everyone dearly. So long, and thanks for everything. I am excited for what comes next — a project that is very personally meaningful to me about which I will share details in due time.
Ilya Sutskever
@ilyasut
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We're announcing, together with @ericschmidt: Superalignment Fast Grants.
$10M in grants for technical research on aligning superhuman AI systems, including weak-to-strong generalization, interpretability, scalable oversight, and more.
Apply by Feb 18! openai.com/blog/superalignment-fast-grants -
Ilya Sutskever reposted
Leopold Aschenbrenner
@leopoldasch
RLHF works great for today's models. But aligning future superhuman models will present fundamentally new challenges.
We need new approaches + scientific understanding.
New researchers can make enormous contributions—and we want to fund you!
Apply by Feb 18!
OpenAI
@OpenAI
We're announcing, together with @ericschmidt: Superalignment Fast Grants.
$10M in grants for technical research on aligning superhuman AI systems, including weak-to-strong generalization, interpretability, scalable oversight, and more.
Apply by Feb 18! openai.com/blog/superalignment-fast-grants -
Ilya Sutskever reposted
Boaz Barak
@boazbaraktcs
My view is that what makes super-alignment "super" is ensuring we can safely scale the capabilities of AIs even though we can't scale their human supervisors. For this, it is imperative to study the "weak teacher strong student" setting. Paper shows great promise in this area!
AK
@_akhaliq
Open AI new paper
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
paper: cdn.openai.com/papers/weak-to-strong-generalization.pdf
blog: openai.com/research/weak-to-strong-generalization
Widely used alignment techniques, such as reinforcement learning from human feedback (RLHF), rely on the ability of humans to supervise model behavior—for example, to evaluate whether a model faithfully followed instructions or generated safe outputs. However, future superhuman models will behave in complex ways too difficult for humans to reliably evaluate; humans will only be able to weakly supervise superhuman models. We study an analogy to this problem: can weak model supervision elicit the full capabilities of a much stronger model? We test this using a range of pretrained language models in the GPT-4 family on natural language processing (NLP), chess, and reward modeling tasks. We find that when we naively finetune strong pretrained models on labels generated by a weak model, they consistently perform better than their weak supervisors, a phenomenon we call weak-to-strong generalization. However, we are still far from recovering the full capabilities of strong models with naive finetuning alone, suggesting that techniques like RLHF may scale poorly to superhuman models without further work. We find that simple methods can often significantly improve weak-to-strong generalization: for example, when finetuning GPT-4 with a GPT-2-level supervisor and an auxiliary confidence loss, we can recover close to GPT-3.5-level performance on NLP tasks. Our results suggest that it is feasible to make empirical progress today on a fundamental challenge of aligning superhuman models. -
Ilya Sutskever reposted
Sam Altman
@sama
i'd particularly like to recognize @CollinBurns4 for today's generalization result, who came to openai excited to pursue this vision and helped get the rest of the team excited about it!
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
Leo Gao
@nabla_theta
new paper! one reason aligning superintelligence is hard is because it will be different from current models, so doing useful empirical research today is hard. we fix one major disanalogy of previous empirical setups. I'm excited for future work making it even more analogous.
OpenAI
@OpenAI
In the future, humans will need to supervise AI systems much smarter than them.
We study an analogy: small models supervising large models.
Read the Superalignment team's first paper showing progress on a new approach, weak-to-strong generalization: openai.com/research/weak-to-strong-generalization -
Ilya Sutskever reposted
Greg Brockman
@gdb
New direction for AI alignment — weak-to-strong generalization.
Promising initial results: we used outputs from a weak model (fine-tuned GPT-2) to communicate a task to a stronger model (GPT-4), resulting in intermediate (GPT-3-level) performance.
OpenAI
@OpenAI
In the future, humans will need to supervise AI systems much smarter than them.
We study an analogy: small models supervising large models.
Read the Superalignment team's first paper showing progress on a new approach, weak-to-strong generalization: openai.com/research/weak-to-strong-generalization -
Ilya Sutskever reposted
Pavel Izmailov
@Pavel_Izmailov
Extremely excited to have this work out, the first paper from the Superalignment team! We study how large models can generalize from supervision of much weaker models.
OpenAI
@OpenAI
In the future, humans will need to supervise AI systems much smarter than them.
We study an analogy: small models supervising large models.
Read the Superalignment team's first paper showing progress on a new approach, weak-to-strong generalization: openai.com/research/weak-to-strong-generalization -
Ilya Sutskever reposted
Jan Leike
@janleike
Kudos especially to @CollinBurns4 for being the visionary behind this work, @Pavel_Izmailov for all the great scientific inquisition, @ilyasut for stoking the fires, @janhkirchner and @leopoldasch for moving things forward every day. Amazing ✨
-
Ilya Sutskever reposted
Jan Leike
@janleike
Super excited about our new research direction for aligning smarter-than-human AI:
We finetune large models to generalize from weak supervision—using small models instead of humans as weak supervisors.
Check out our new paper:
openai.com/research/weak-to-strong-generalization -
Ilya Sutskever reposted
OpenAI
@OpenAI
In the future, humans will need to supervise AI systems much smarter than them.
We study an analogy: small models supervising large models.
Read the Superalignment team's first paper showing progress on a new approach, weak-to-strong generalization: openai.com/research/weak-to-strong-generalization -
Ilya Sutskever
@ilyasut
❤️
-
Ilya Sutskever reposted
OpenAI
@OpenAI
Sam Altman is back as CEO, Mira Murati as CTO and Greg Brockman as President. OpenAI has a new initial board. Messages from @sama and board chair @btaylor openai.com/blog/sam-altman-returns-as-ceo-openai-has-a-ne…
-
Ilya Sutskever
@ilyasut
There exists no sentence in any language that conveys how happy I am:
-
Ilya Sutskever reposted
Greg Brockman
@gdb
Returning to OpenAI & getting back to coding tonight.
-
Ilya Sutskever reposted
Mira Murati
@miramurati
🚢ing
-
Ilya Sutskever reposted
Sam Altman
@sama
i love openai, and everything i’ve done over the past few days has been in service of keeping this team and its mission together. when i decided to join msft on sun evening, it was clear that was the best path for me and the team. with the new board and w satya’s support, i’m looking forward to returning to openai, and building on our strong partnership with msft.
-
Ilya Sutskever
@ilyasut
I deeply regret my participation in the board's actions. I never intended to harm OpenAI. I love everything we've built together and I will do everything I can to reunite the company.
-
Ilya Sutskever
@ilyasut
if you value intelligence above all other human qualities, you’re gonna have a bad time
-
Ilya Sutskever
@ilyasut
Just stumbled upon a pretty good abbreviation to “just ask chatgpt”: chask it!
-
Ilya Sutskever
@ilyasut
Ego is the enemy of growth
-
Ilya Sutskever reposted
Aditya Ramesh
@model_mechanic
The DALL·E team is hiring researchers who are passionate about pushing the frontiers of generative modeling and responsibly getting this technology in the hands of people. Some of our best hires have been people who don't have formal research backgrounds or graduate degrees, but have demonstrated exceptional achievement in other ways, e.g. through personal projects or startups. If you're interested, please DM me a link to your work!
-
Ilya Sutskever
@ilyasut
In the future, once the robustness of our models will exceed some threshold, we will have *wildly effective* and dirt cheap AI therapy. Will lead to a radical improvement in people’s experience of life. One of the applications I’m most eagerly awaiting.
Lilian Weng
@lilianweng
Just had a quite emotional, personal conversation w/ ChatGPT in voice mode, talking about stress, work-life balance. Interestingly I felt heard & warm. Never tried therapy before but this is probably it? Try it especially if you usually just use it as a productivity tool.
-
Ilya Sutskever
@ilyasut
Empathy in life and business is underrated
-
Ilya Sutskever reposted
Christine
@mcleavey
Excited to bring voice mode to ChatGPT! Amazing work by @AlecRad @_jongwook_kim @txhf @giertler @michpokrass @joannejang and many others!
-
Ilya Sutskever
@ilyasut
Wagmi
-
Ilya Sutskever reposted
Viral Vora
@viralvora_
Just heard Jan's podcast on AI alignment. Such an underrated and important talk. Even an absolute beginner could understand. Happy to see AI alignment seems to be in the right hands.
@janleike @ilyasut -
Ilya Sutskever reposted
Greg Brockman
@gdb
New skin for the OpenAI cookbook, our open-source collection of examples and guides for building with the OpenAI API.
Easier to use and discover than ever before: cookbook.openai.com
Great work led by @simonpfish. -
Ilya Sutskever reposted
OpenAI
@OpenAI
Announcing the OpenAI Learning Impact Prize in partnership with @ToolsCompete. $100,000 cash prize for the top winners, plus API credits for runner-ups. Open to any startup or individual with an innovative idea for AI in education:
Tools Competition
@ToolsCompete
We are thrilled to partner with @OpenAI to launch 🚀 the OpenAI Learning Impact Prize. Winners will receive $100,000, API credits, and technical guidance to scale their solutions leveraging AI for impact in education. Submit your abstract by November 10 tools-competition.org
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
Peter Welinder
@npew
We just announced DALL·E 3, our most powerful generative image model. Coming soon to ChatGPT Plus & Enterprise.
OpenAI
@OpenAI
Our new text-to-image model, DALL·E 3, can translate nuanced requests into extremely detailed and accurate images.
Coming soon to ChatGPT Plus & Enterprise, which can help you craft amazing prompts to bring your ideas to life:
openai.com/dall-e-3 -
Ilya Sutskever reposted
Noam Brown
@polynoamial
Dalle 3 is coming out! I've been having a lot of fun playing around with it internally
OpenAI
@OpenAI
Our new text-to-image model, DALL·E 3, can translate nuanced requests into extremely detailed and accurate images.
Coming soon to ChatGPT Plus & Enterprise, which can help you craft amazing prompts to bring your ideas to life:
openai.com/dall-e-3 -
Ilya Sutskever reposted
Andrej Karpathy
@karpathy
Very nice work on DALL·E 3 (openai.com/dall-e-3) by @model_mechanic and the team.
The ChatGPT UI/UX is quite nice because it does a lot of the prompt engineering for you, you just direct it on a high level and ask for variations simply and in words. -
Ilya Sutskever
@ilyasut
Incredible work by @model_mechanic and team!
-
Ilya Sutskever
@ilyasut
AGI pilled-ness is a spectrum
-
Ilya Sutskever
@ilyasut
Practical alignment work is both critically important and immediately impactful. Please consider applying:
Jan Leike
@janleike
If you're into practical alignment, consider applying to @lilianweng's team. They're building some really exciting stuff:
- Automatically extract intent from a fine-tuning dataset
- Make models robust to jailbreaks
- Detect & mitigate harmful use
- ...
www.linkedin.com/feed/update/urn:li:activity:708292803621… -
Ilya Sutskever reposted
OpenAI
@OpenAI
🚀 Registration to attend OpenAI DevDay in-person is now live! Apply here: devday.openai.com
-
Ilya Sutskever
@ilyasut
I feel fortunate that a place like the Bay Area exists
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We’re opening an office in Dublin, Ireland 🇮🇪 openai.com/blog/introducing-openai-dublin
-
Ilya Sutskever reposted
Greg Brockman
@gdb
There's something really great about working closely with the same set of people for many years, but I'm also really loving working with the new leaders & contributors who have joined @OpenAI recently.
-
Ilya Sutskever
@ilyasut
Congratulations to my cofounder @gdb for being featured in the Time 100 AI list! It’s been a massive privilege to have worked together with you all these years! time.com/collection/time100-ai/6309033/greg-brockman/
-
Ilya Sutskever reposted
Sam Altman
@sama
on november 6, we’ll have some great stuff to show developers! (no gpt-5 or 4.5 or anything like that, calm down, but still i think people will be very happy…)
www.openai.com/blog/announcing-openai-devday -
Ilya Sutskever reposted
OpenAI
@OpenAI
We’ll be hosting our first developer conference, OpenAI DevDay, on November 6. Registration to attend in person in San Francisco will open in a few weeks. We’ll also livestream the keynote. www.openai.com/blog/announcing-openai-devday
-
Ilya Sutskever
@ilyasut
The perfect has destroyed much perfectly good good
-
Ilya Sutskever reposted
Richard Ngo
@RichardMCNgo
Earlier this year I helped organize the SF Alignment Workshop, which brought together top alignment and mainstream ML researchers to discuss and debate alignment risks and research directions. There were many great talks, which we’re excited to share now - see thread.
-
Ilya Sutskever
@ilyasut
I feel blessed and grateful to be working with my colleagues
-
Ilya Sutskever
@ilyasut
Can’t let your mode collapse
-
Ilya Sutskever reposted
Pieter Abbeel
@pabbeel
Hard to imagine a better way for me to flash right back into @OpenAI early days 2016, inspiring clarity of thinking and communication by @ilyasut
www.youtube.com/live/AKMuA_TVz3A -
Ilya Sutskever
@ilyasut
It’s a mistake to let an uncertain future ruin the present
-
Ilya Sutskever reposted
Jan Leike
@janleike
Great conversation with @robertwiblin on how alignment is one of the most interesting ML problems, what the Superalignment Team is working on, what roles we're hiring for, what's needed to reach an awesome future, and much more
👇 Check it out 👇
80000hours.org/podcast/episodes/jan-leike-superalignment/ -
Ilya Sutskever
@ilyasut
Great Potential: You!
-
Ilya Sutskever reposted
Diogo Almeida
@CompleteSkeptic
-
Ilya Sutskever
@ilyasut
Wrong motivation -> wrong results
-
Ilya Sutskever
@ilyasut
Criticizing a decision is, to a first order approximation, 100x easier than making one
-
Ilya Sutskever
@ilyasut
The biggest trick our brains play is to create the illusion of understanding what’s going on, both around us and in the world
-
Ilya Sutskever
@ilyasut
Beware of ideas that, as a consequence of believing in them, make you feel superior to other people
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We are partnering with @JournalismProj to explore how AI tools can assist local newsrooms: openai.com/blog/partnership-with-american-journalism-proj…
-
Ilya Sutskever reposted
Sam Altman
@sama
makes me happy how much people love code interpreter!
-
Ilya Sutskever reposted
Greg Brockman
@gdb
there is a unique kind of joy that comes from being part of a focused team solving hard problems
-
Ilya Sutskever
@ilyasut
Let the sword of reason shine
-
Ilya Sutskever reposted
Sam Altman
@sama
it is very disappointing to see the FTC's request start with a leak and does not help build trust.
that said, it’s super important to us that out technology is safe and pro-consumer, and we are confident we follow the law. of course we will work with the FTC. -
Ilya Sutskever reposted
Lilian Weng
@lilianweng
(1/3) Alongside Superalignment team, my team is working on the practical side of alignment: Building systems to enable safe AI deployment. We are looking for strong research engineers and scientists to join the efforts.
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
Jan Leike
@janleike
Our new goal is to solve alignment of superintelligence within the next 4 years.
OpenAI is committing 20% of its compute to date towards this goal.
Join us in researching how to best spend this compute to solve the problem!
openai.com/blog/introducing-superalignment -
Ilya Sutskever reposted
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We need new technical breakthroughs to steer and control AI systems much smarter than us.
Our new Superalignment team aims to solve this problem within 4 years, and we’re dedicating 20% of the compute we've secured to date towards this problem.
Join us! openai.com/blog/introducing-superalignment/ -
Ilya Sutskever
@ilyasut
What do people and artificial neural networks agree on?
That attention is all you need -
Ilya Sutskever reposted
OpenAI
@OpenAI
In May and June, we traveled to 25 cities across 6 continents to better understand how users, developers, and government leaders are thinking about the creation and deployment of safe AI — from today’s AI to superintelligence. Here’s what’s next: openai.com/blog/insights-from-global-conversations/
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We are excited to announce OpenAI's first international expansion with a new office in London! 🇬🇧 openai.com/blog/introducing-openai-london
-
Ilya Sutskever
@ilyasut
A one sentence articulation (of existing ideas) for why AI alignment need not be straightforward:
humans can lie, hide their intent (alignment), and do so for years. Why not AGI? This can be hard to detect. -
Ilya Sutskever
@ilyasut
GPU = new Bitcoin
-
Ilya Sutskever
@ilyasut
There’s an amusing anti correlation between networking and actually working
-
Ilya Sutskever reposted
OpenAI
@OpenAI
GPT-4 and GPT-3.5 Turbo models in the API now support calling your custom functions, allowing the model to use tools you design for it. Also — reduced pricing & new model versions (including 16k context for 3.5 Turbo): openai.com/blog/function-calling-and-other-api-updates
-
Ilya Sutskever reposted
Barret Zoph
@barret_zoph
Our team at OpenAI is hiring! We're looking for engineers/researchers who do rigorous and thoughtful work understanding and evaluating LLMs like ChatGPT.
If you're interested, please apply online and DM me with work that you've done! -
Ilya Sutskever reposted
OpenAI
@OpenAI
Announcing the Cybersecurity Grant Program — a $1M initiative to boost and quantify AI-powered cybersecurity capabilities and to foster high-level AI and cybersecurity discourse: openai.com/blog/openai-cybersecurity-grant-program
-
Ilya Sutskever reposted
Jan Leike
@janleike
Really interesting result on using LLMs to do math:
Supervising every step works better than only checking the answer.
Some thoughts how this matters for alignment 👇
openai.com/research/improving-mathematical-reasoning-with… -
Ilya Sutskever reposted
Andrej Karpathy
@karpathy
yay the ability to share ChatGPT conversations is now rolling out. I can share a few favorites.
E.g. GPT-4 is great at generating Anki flash cards, helping you to memorize any document. Example:
chat.openai.com/share/eef34fe5-0c8e-4595-9c28-2e9f05f05393
Easy to then import in Anki: apps.ankiweb.net -
Ilya Sutskever reposted
Greg Brockman
@gdb
GPT-4 for personal tutoring: www.reddit.com/r/ChatGPT/comments/13sj8os/chatgpt_is_savi…
-
Ilya Sutskever
@ilyasut
First step towards a democratically controlled AI
OpenAI
@OpenAI
We're launching ten $100,000 grants for building prototypes of a democratic process for steering AI. Our goal is to fund experimentation with methods for gathering nuanced feedback from everyone on how AI should behave. Apply by June 24, 2023: openai.com/blog/democratic-inputs-to-ai
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
Sam Altman
@sama
something like an IAEA for advanced AI is worth considering, and the shape of the tech may make it feasible:
openai.com/blog/governance-of-superintelligence
(and to make this harder to willfully misinterpret: it's important that any such regulation not constrain AI below a high capability threshold) -
Ilya Sutskever reposted
Sam Altman
@sama
had a great first week of the openai world tour in toronto, DC, rio, lagos, and lisbon.
fun to see what people are building and get (lots of) feature requests, and even fun to talk to policymakers!
madrid, warsaw, paris, london, and munich this week. -
Ilya Sutskever reposted
OpenAI
@OpenAI
Initial ideas for governance of superintelligence, including forming an international oversight organization for future AI systems much more capable than any today: openai.com/blog/governance-of-superintelligence
-
Ilya Sutskever reposted
Sam Altman
@sama
regulation should take effect above a capability threshold.
AGI safety is really important, and frontier models should be regulated.
regulatory capture is bad, and we shouldn't mess with models below the threshold. open source models and small startups are obviously important. -
Ilya Sutskever reposted
-
Ilya Sutskever
@ilyasut
I love this app’s speech recognition. As someone with an accent that confuses my phone’s speech recognition, it is a real joy to speak to the app and to be fully understood, every time.
OpenAI
@OpenAI
Introducing the ChatGPT app for iOS! We’re live in the US and will expand to additional countries in the coming weeks. Android is next! openai.com/blog/introducing-the-chatgpt-app-for-ios
-
Ilya Sutskever reposted
OpenAI
@OpenAI
Introducing the ChatGPT app for iOS! We’re live in the US and will expand to additional countries in the coming weeks. Android is next! openai.com/blog/introducing-the-chatgpt-app-for-ios
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We’re rolling out web browsing and Plugins to all ChatGPT Plus users over the next week! Moving from alpha to beta, they allow ChatGPT to access the internet and to use 70+ third-party plugins. help.openai.com/en/articles/6825453-chatgpt-release-notes
-
Ilya Sutskever reposted
Sam Altman
@sama
microsoft becomes helion's first customer, in the first commercial deal for fusion power:
www.wsj.com/articles/microsoft-bets-that-fusion-power-is-… -
Ilya Sutskever reposted
Greg Brockman
@gdb
We're using GPT-4 to interpret the neurons in GPT-2.
Step towards our alignment plan of using AI to automate alignment research (openai.com/blog/our-approach-to-alignment-research).
GPT-2 neuron map released here: openaipublic.blob.core.windows.net/neuron-explainer/neuro…
OpenAI
@OpenAI
We applied GPT-4 to interpretability — automatically proposing explanations for GPT-2's 300k neurons — and found neurons responding to concepts like similes, “things done correctly,” or expressions of certainty. We aim to use Al to help us understand Al: openai.com/research/language-models-can-explain-neurons-i…
-
Ilya Sutskever reposted
Jan Leike
@janleike
Really exciting new work on automated interpretability:
We ask GPT-4 to explain firing patterns for individual neurons in LLMs and score those explanations.
openai.com/research/language-models-can-explain-neurons-i… -
Ilya Sutskever
@ilyasut
The meaning of life is neural net training
(parenting is a special case) -
Ilya Sutskever reposted
Kareem Carr
@kareem_carr
This is the greatest mathematician story ever told
-
Ilya Sutskever
@ilyasut
It’s not a religion if it’s true
-
Ilya Sutskever reposted
Srinivas Narayanan
@snsf
Joined OpenAI 10 days ago. This place is absolutely electric and very special. So excited to be on this journey to advance AI and make it useful for all of humanity with @sama, @miramurati, @gdb, @ilyasut, @npew, @bobmcgrewai and the whole OpenAI team.
-
Ilya Sutskever reposted
OpenAI
@OpenAI
ChatGPT users can now turn off chat history, allowing you to choose which conversations can be used to train our models: openai.com/blog/new-ways-to-manage-your-data-in-chatgpt
-
Ilya Sutskever reposted
Greg Brockman
@gdb
TED talk from earlier this week. Shows a bit of the future of AI tools, how we teach AIs to follow our intent, and how the tools themselves can help scale our ability to give high-quality feedback: www.ted.com/talks/greg_brockman_the_inside_story_of_chatg…
-
Ilya Sutskever
@ilyasut
The Ray of compression shines brightly
-
Ilya Sutskever reposted
Greg Brockman
@gdb
GPT-4 for saving Terence Tao's time: mathstodon.xyz/@tao/110172426733603359
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We're launching the OpenAI Bug Bounty Program — earn cash awards for finding & responsibly reporting security vulnerabilities. openai.com/blog/bug-bounty-program
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We’re sharing details on our approach to safety openai.com/blog/our-approach-to-ai-safety
-
Ilya Sutskever
@ilyasut
Powerful and non obvious scientific ideas, once internalized, usually become blindingly obvious.
This makes it hard to appreciate just how innovative a well known idea has been in the past. -
Ilya Sutskever
@ilyasut
This too shall pass
-
Ilya Sutskever reposted
Dripped Out Technology Brothers
@TechBroDrip
Mira Murati (CTO of OpenAI) + Sam Altman (CEO of OpenAI) + Greg Brockman (President of OpenAI) + Ilya Sutskever (Chief Scientist of OpenAI)
-
Ilya Sutskever reposted
Peter Welinder
@npew
The ChatGPT browsing plugin is useful for answering questions, doing research, and finding up-to-date information.
-
Ilya Sutskever reposted
Stephen Wolfram
@stephen_wolfram
2.5 months later...in collaboration with @openAI , ChatGPT gets its "Wolfram superpowers"!
writings.stephenwolfram.com/2023/03/chatgpt-gets-its-wolf… -
Ilya Sutskever reposted
OpenAI
@OpenAI
We are adding support for plugins to ChatGPT — extensions which integrate it with third-party services or allow it to access up-to-date information. We’re starting small to study real-world use, impact, and safety and alignment challenges: openai.com/blog/chatgpt-plugins
-
Ilya Sutskever reposted
Sam Altman
@sama
we didn’t realize how important code-davinci-002 was to researchers, so we are keeping it going in our researcher access program: openai.com/form/researcher-access-program
we are also providing researcher access to the base GPT-4 model! -
Ilya Sutskever reposted
NVIDIA GTC
@NVIDIAGTC
Don't miss the fireside chat live in 30 minutes with @OpenAI's co-founder and chief scientist @ilyasut and our founder and CEO, Jensen Huang.
Join us to hear their perspectives on #ai and a vision for the future. nvda.ws/3LHqbdq -
Ilya Sutskever reposted
Logan Kilpatrick
@OfficialLoganK
We underestimated how much some of y’all love Codex. 💙 We get it now; we will continue to support it via our research access program. Thanks for the feedback!
Soon, we will also add rate-limited GPT-4 base model access to the researcher access program.
openai.com/form/researcher-access-program -
Ilya Sutskever reposted
OpenAI
@OpenAI
Great news! ChatGPT Plus subscriptions are now available in India. Get early access to new features, including GPT-4 today: chat.openai.com
-
Ilya Sutskever
@ilyasut
One of life’s great pleasures is to work with brilliant and kind colleagues
-
Ilya Sutskever reposted
Geoffrey Hinton
@geoffreyhinton
Reinforcement Learning by Human Feedback is just parenting for a supernaturally precocious child.
-
Ilya Sutskever reposted
Boris Power
@BorisMPower
Two ways to get GPT-4 access 🤖
1. Sign up to the waitlist (it’ll take a few weeks)🤞
2. Contribute an open source eval (instant access if the PR is approved!) 🎉 -
Ilya Sutskever reposted
Miles Brundage
@Miles_Brundage
My colleagues did such great jobs with the blog post, technical report, system card, video, livestream, etc. that I don't have much to add re: GPT-4.
So I'll just say I'm fortunate to have such talented colleagues, check out our work, and we're hiring! openai.com/research/gpt-4 -
Ilya Sutskever reposted
Mark Chen
@markchen90
GPT-4 shipped today! It sets a new high water mark on capabilities and alignment, but don't forget that it's also a powerful multimodal model that's SoTA on a number of VQA tasks, even compared to fine-tuned baselines. It can explain memes!
OpenAI
@OpenAI
Announcing GPT-4, a large multimodal model, with our best-ever results on capabilities and alignment: openai.com/product/gpt-4
-
Ilya Sutskever reposted
Greg Brockman
@gdb
GPT-4 demo livestream replay: www.youtube.com/watch?v=outcGtbnMuQ
-
Ilya Sutskever reposted
Greg Brockman
@gdb
GPT-4 was, in many ways, our first whole-company project: openai.com/contributions/gpt-4. Jakub did a stellar job as lead for pretraining.
Sam Altman
@sama
GPT-4 was truly a team effort from our entire company, but the overall leadership and technical vision of Jakub Pachocki for the pretraining effort was remarkable and we wouldn’t be here without it
-
Ilya Sutskever reposted
Be My Eyes
@BeMyEyes
We are thrilled to present Virtual Volunteer™, a digital visual assistant powered by @OpenAI’s GPT-4 language model. Virtual Volunteer will answer any question about an image and provide instantaneous visual assistance in real-time within the app. #accessibility #inclusion #csun
-
Ilya Sutskever reposted
Chelsea Sierra Voss
@csvoss
GPT-4 is very impressive at complex reasoning tasks. Pay close attention to these numbers.
Reaching the top 0.5% on the USA Biolympiad Semifinal Exam is on par with – or better than! – me, when I trained for that very exam as a gold medalist at my peak.
Try it out today!
Nick Turley
@nickaturley
GPT-4 is here. It can recognize images and outperforms at complex tasks. Some highlights:
- 5 on AP Bio
- Top 10% on the Bar
- Top 0.5% on the US Bio Olympiad.
Best part? We're rolling out to to ChatGPT Plus users today — no waitlist.
openai.com/research/gpt-4 #gpt4 #chatgpt -
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
wild generalization of GPT-4
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
Jack Rae
@jack_w_rae
GPT-4 is up!
Trained, aligned, evaluated & served by an incredible group of people 💙
openai.com/research/gpt-4 -
Ilya Sutskever reposted
Khan Academy
@khanacademy
Did you hear? 👀 Khan Academy is using GPT-4 from @OpenAI to shape the future of learning.
Starting today, you can sign up to test our AI-powered guide, Khanmigo. A tutor for learners. An assistant for teachers.
Come explore with us! ✨ -
Ilya Sutskever reposted
Sam Altman
@sama
GPT-4 was truly a team effort from our entire company, but the overall leadership and technical vision of Jakub Pachocki for the pretraining effort was remarkable and we wouldn’t be here without it
-
Ilya Sutskever reposted
Peter Welinder
@npew
GPT-4 can read images. Still in research preview, but the team is working hard on getting it ready for broader access.
-
Ilya Sutskever reposted
Peter Welinder
@npew
GPT-4 scores in the 90th percentile on the bar exam. And crushes it on many other tests.
-
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
I am extremely proud of what the hundreds of extremely talented, hardworking, caring people at OpenAI have built.
Welcome GPT-4 to the world! We are happy to have you here.
You will entertain us, bring us tears, make us laugh, and help us. Thank you.
openai.com/research/gpt-4 -
Ilya Sutskever reposted
OpenAI
@OpenAI
Join us at 1 pm PT today for a developer demo livestream showing GPT-4 and its capabilities/limitations: youtube.com/live/outcGtbnMuQ
(comments in Discord: discord.gg/openai) -
Ilya Sutskever reposted
Andrej Karpathy
@karpathy
🎉 GPT-4 is out!!
- 📈 it is incredible
- 👀 it is multimodal (can see)
- 😮 it is on trend w.r.t. scaling laws
- 🔥 it is deployed on ChatGPT Plus: chat.openai.com
- 📺 watch the developer demo livestream at 1pm: youtube.com/live/outcGtbnMuQ
OpenAI
@OpenAI
Announcing GPT-4, a large multimodal model, with our best-ever results on capabilities and alignment: openai.com/product/gpt-4
-
Ilya Sutskever reposted
OpenAI
@OpenAI
Announcing GPT-4, a large multimodal model, with our best-ever results on capabilities and alignment: openai.com/product/gpt-4
-
Ilya Sutskever
@ilyasut
One of my hopes for the good AGI future is that people will become kinder to each other
-
Ilya Sutskever
@ilyasut
now 10x cheaper!
OpenAI
@OpenAI
ChatGPT and Whisper are now available through our API (plus developer policy updates). We ❤️ developers: openai.com/blog/introducing-chatgpt-and-whisper-apis
-
Ilya Sutskever reposted
Peter Welinder
@npew
We’re releasing ChatGPT in our API. This allows anyone to build AI applications powered by ChatGPT. After many optimizations, we’re pricing it at a 10x lower price than the previous GPT-3.5 model, to allow for many more use cases than before.
openai.com/blog/introducing-chatgpt-and-whisper-apis -
Ilya Sutskever reposted
Greg Brockman
@gdb
ChatGPT API now available, 10% the price of our flagship language model & matching/better at any pretty much any task (not just chat).
Also released Whisper API & greatly improved our developer policies in response to feedback. We ❤️ developers: openai.com/blog/introducing-chatgpt-and-whisper-apis -
Ilya Sutskever reposted
OpenAI
@OpenAI
ChatGPT and Whisper are now available through our API (plus developer policy updates). We ❤️ developers: openai.com/blog/introducing-chatgpt-and-whisper-apis
-
Ilya Sutskever reposted
Sam Altman
@sama
ChatGPT has an ambitious roadmap and is bottlenecked by engineering. pretty cool stuff is in the pipeline!
want to be stressed, watch some GPUs melt, and have a fun time? good at doing impossible things?
send evidence of exceptional ability to chatgpt-eng@openai.com -
Ilya Sutskever reposted
OpenAI
@OpenAI
How we are planning for AGI: openai.com/blog/planning-for-agi-and-beyond
-
Ilya Sutskever
@ilyasut
All you need is to be less perplexed
-
Ilya Sutskever
@ilyasut
There’s a possibility that jailbreaks that work on advanced jailbreak-resistant models will also work on people 🤔
-
Ilya Sutskever reposted
Sam Altman
@sama
our current thoughts on hard questions about how AI systems should behave:
1) less biased defaults, 2) lots of user customization within very broad bounds, 3) public input on bounds and defaults
openai.com/blog/how-should-ai-systems-behave/ -
Ilya Sutskever reposted
OpenAI
@OpenAI
Information on ChatGPT’s alignment, plans to improve it, giving users more control, and early thoughts on public input: openai.com/blog/how-should-ai-systems-behave/
-
Ilya Sutskever
@ilyasut
Many believe that great AI advances must contain a new “idea”. But it is not so: many of AI’s greatest advances had the form “huh, turns out this familiar unimportant idea, when done right, is downright incredible”
-
Ilya Sutskever reposted
Satya Nadella
@satyanadella
Bing and Edge + AI: a new way to search starts today blogs.microsoft.com/blog/2023/02/07/reinventing-search-wi…
-
Ilya Sutskever reposted
Dan Siroker
@dsiroker
😂 Business Insider thinks OpenAI’s bold vision to bring AGI into reality means “adjusted gross income”
-
Ilya Sutskever
@ilyasut
must reach strong AGI (adjusted gross income)
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We are piloting ChatGPT Plus, a subscription plan that offers faster response times and reliability during peak hours. And of course, the free tier of ChatGPT is still available. openai.com/blog/chatgpt-plus/
-
Ilya Sutskever reposted
NVIDIA GTC
@NVIDIAGTC
Get excited for our upcoming GTC! You won't want to miss this fireside chat with OpenAI's @ilyasut and NVIDIA's Jensen Huang. Join us March 22 online to hear their perspectives on AI and a vision of the future. Register for free today: nvda.ws/3HKrXbn #gtc23
-
Ilya Sutskever
@ilyasut
Denial is a hell of a drug
-
Ilya Sutskever reposted
Sam Altman
@sama
i know im not supposed to brag about openai, but the talent density at this scale (375 people) is 🤯 and i dont think has happened in the tech industry in recent memory
-
Ilya Sutskever reposted
Satya Nadella
@satyanadella
In this next phase of our partnership with @OpenAI, we will deliver the best AI infrastructure, models, and toolchain for customers to safely and responsibly build and run their applications on Azure. blogs.microsoft.com/blog/2023/01/23/microsoftandopenaiext…
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We are happy to announce the next phase of our partnership with Microsoft: openai.com/blog/openai-and-microsoft-extend-partnership/
-
Ilya Sutskever
@ilyasut
-
Ilya Sutskever reposted
Andrej Karpathy
@karpathy
Plan is to throw a party in the Andromeda galaxy 1B years from now. Everyone welcome, except for those who litter
-
Ilya Sutskever
@ilyasut
If an explanation is long, there’s a high chance that it’s wrong
-
Ilya Sutskever
@ilyasut
Happy new year! May all our AGIs love humanity
-
Ilya Sutskever
@ilyasut
Every government, no matter how democratic, has a little Soviet heart deep within
-
Ilya Sutskever reposted
Greg Brockman
@gdb
Interesting first results of applying GPT embeddings to early diagnosis of dementia — 80% accuracy on a challenge dataset of speech recordings. Note also the paper was submitted in August; we've released much better embeddings since then! eurekalert.org/news-releases/975246
-
Ilya Sutskever reposted
Sam Altman
@sama
unintended; going to take us some time to get all of this right (and it still requires more research).
generally speaking, within very wide bounds we want to enable people get the behavior they want when using AI. will talk more about it in january! -
Ilya Sutskever reposted
Richard Ngo
@RichardMCNgo
At OpenAI a lot of our work aims to align language models like ChatGPT with human preferences. But this could become much harder once models can act coherently over long timeframes and exploit human fallibility to get more reward.
📜Paper: arxiv.org/abs/2209.00626
🧵Thread: -
Ilya Sutskever reposted
Sherwin Wu
@sherwinwu
We just open sourced our tokenizer! This should help all users of our API better understand their usage. It's also incredibly fast – 3-6x faster than other open source alternatives!
github.com/openai/tiktoken -
Ilya Sutskever reposted
Greg Brockman
@gdb
Just released a new embedding model which outperforms our previous best one while being 99.8% cheaper: openai.com/blog/new-and-improved-embedding-model/
-
Ilya Sutskever reposted
Boris Power
@BorisMPower
We just released new embeddings at a ridiculously low price!
For $40 one can embed all Joe Rogan podcast's transcripts (100k pages of text!) - who will go viral by building a semantic search for it?
This is better, and 150x cheaper than previous models! openai.com/blog/new-and-improved-embedding-model -
Ilya Sutskever reposted
OpenAI
@OpenAI
Our new embedding model is significantly more capable at language processing and code tasks, cost effective, and simpler to use. openai.com/blog/new-and-improved-embedding-model/
-
Ilya Sutskever
@ilyasut
Machine learning is just statistics. On steroids. Lots and lots of steroids.
-
Ilya Sutskever reposted
Sam Altman
@sama
ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness.
it's a mistake to be relying on it for anything important right now. it’s a preview of progress; we have lots of work to do on robustness and truthfulness. -
Ilya Sutskever reposted
Microsoft Azure
@Azure
We appreciate the love 💙 We work hard to keep our users happy, and our partnerships strong, thanks for noticing 😉
-
Ilya Sutskever reposted
Sam Altman
@sama
🙏 to my colleagues for making it feel like i get to walk into the modern bell labs every day.
please consider joining us at openai! being in the room for research breakthroughs is awesome, the problems are interesting, the people are great, and we ship.
youtu.be/AyOnug-3OKM -
Ilya Sutskever reposted
Mira Murati
@miramurati
In just 5 days since launch, ChatGPT has reached over 1M users. Thank you for your feedback to help us improve the platform!
-
Ilya Sutskever reposted
Greg Brockman
@gdb
Microsoft partnership has been one of OpenAI's secrets to success — we work very closely with Azure to produce AI training & serving infrastructure that can scale to our cutting-edge (& entirely unprecedented!) needs.
Sam Altman
@sama
microsoft, and particularly azure, don’t get nearly enough credit for the stuff openai launches. they do an amazing amount of work to make it happen; we are deeply grateful for the partnership. 🙏 they have built by far the best AI infra out there.
-
Ilya Sutskever reposted
Jan Leike
@janleike
New blog post on why I'm excited about OpenAI's approach to alignment, including some responses to common objections:
aligned.substack.com/p/alignment-optimism -
Ilya Sutskever reposted
Sam Altman
@sama
microsoft, and particularly azure, don’t get nearly enough credit for the stuff openai launches. they do an amazing amount of work to make it happen; we are deeply grateful for the partnership. 🙏 they have built by far the best AI infra out there.
-
Ilya Sutskever reposted
Sam Altman
@sama
ChatGPT launched on wednesday. today it crossed 1 million users!
-
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
Many might not know.
@johnschulman2 is the main person driving ChatGPT, and I would like to congratulate him and the team!
Worth to follow. -
Ilya Sutskever reposted
OpenAI
@OpenAI
“Evolution of Man” — Karen X. Cheng x DALL·E @karenxcheng
-
Ilya Sutskever reposted
Sam Altman
@sama
interesting watching people start to debate whether powerful AI systems should behave in the way users want or their creators intend.
the question of whose values we align these systems to will be one of the most important debates society ever has. -
Ilya Sutskever reposted
Sam Altman
@sama
a lot of what people assume is us censoring ChatGPT is in fact us trying to stop it from making up random facts.
tricky to get the balance right with the current state of the tech.
it will get better over time, and we will use your feedback to improve it. -
Ilya Sutskever reposted
Sam Altman
@sama
iterative deployment is, imo, the only safe path and the only way for people, society, and institutions to have time to update and internalize what this all means.
-
Ilya Sutskever reposted
OpenAI
@OpenAI
Try talking with ChatGPT, our new AI system which is optimized for dialogue. Your feedback will help us improve it. openai.com/blog/chatgpt/
-
Ilya Sutskever
@ilyasut
The transformer is well named, as it transformed everything
-
Ilya Sutskever reposted
NVIDIA AI Developer
@NVIDIAAIDev
🏆 Test of Time Award #neurips2022 panel selected “ImageNet Classification with Deep Convolutional Neural Networks” for the lasting and dramatic impact on the machine learning and #ai community. nvda.ws/3gnxk5r
Congrats Alex Krizhevsky, @ilyasut and @geoffreyhinton -
Ilya Sutskever reposted
Kosta Derpanis
@CSProfKGD
#neurips2022 Test of Time Award goes to the "AlexNet paper" - “ImageNet Classification with Deep Convolutional Neural Networks” by Alex Krizhevsky, @ilyasut and @geoffreyhinton
Also known for the most famous incomplete figure ever! 🤓
blog.neurips.cc/2022/11/21/announcing-the-neurips-2022-aw… -
Ilya Sutskever
@ilyasut
The S of our collective LSTM is quite short
-
Ilya Sutskever
@ilyasut
Would be cool to implement dasher using today’s LMs:
youtu.be/0d6yIquOKQ0 -
Ilya Sutskever
@ilyasut
I find it interesting that evolution applies to all “structures”, not just living things: ideologies, organizations, management structures, technologies, etc
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We’ve just launched the DALL·E API so developers can integrate DALL·E directly into their own apps and products. openai.com/blog/dall-e-api-now-available-in-public-beta/
-
Ilya Sutskever
@ilyasut
the biggest obstacle to seeing clearly is the belief that one already sees clearly
-
Ilya Sutskever
@ilyasut
-
Ilya Sutskever
@ilyasut
Deep learning is based on the audacious conjecture that biological neurons and artificial neurons are not that different. Its success to date is evidence for this belief
-
Ilya Sutskever reposted
Boris Power
@BorisMPower
OpenAI was able to strongly relax our policies by:
1. aligning our models to be great at following instructions openai.com/blog/instruction-following/
2. releasing a moderation endpoint, to help developers beta.openai.com/docs/guides/moderation
Almost anything net-good is allowed now! beta.openai.com/docs/usage-policies/use-case-policy
Chris Frantz
@frantzfries
The reason we’re seeing a resurgence in the GPT-3 apps is because OpenAI took the training wheels off.
No more restrictions around long-form writing or social posting using their api *and* you don’t need to be manually approved.
You can pretty much do anything now. -
Ilya Sutskever
@ilyasut
I wonder which insights from developmental psychology will apply to our future NNs
-
Ilya Sutskever reposted
Spencer Greenberg 🔍
@SpencrGreenberg
I was excited to have Ilya Sutskever (@ilyasut), co-founder and Chief Scientist of OpenAI, on my podcast!
We discuss what, if anything, current AI systems "understand" and many other topics related to modern AI.
You can listen to the full episode here:
clearerthinkingpodcast.com/episode/128 -
Ilya Sutskever
@ilyasut
It feels safe to stress out and vulnerable/scary to relax. On why the former is the default
-
Ilya Sutskever
@ilyasut
“prompting” is a transitory term that’s relevant only thanks to flaws in our models
-
Ilya Sutskever
@ilyasut
it's not the worst for an AGI effort to contribute to a future plurality of AGIs all of whom love humanity
-
Ilya Sutskever reposted
The Daily Show
@TheDailyShow
TONIGHT: OpenAI CTO @miramurati is here to talk about DALL·E 2 and the power of AI!
-
Ilya Sutskever
@ilyasut
Human culture is critical civilization-enabling infrastructure. One that’s hard to improve, easy to destroy.
-
Ilya Sutskever
@ilyasut
I find it funny when people say that evolution is “stupid”, when you consider that it’s by far the most intelligent force that can exist in our universe
-
Ilya Sutskever
@ilyasut
California is the America of America
-
Ilya Sutskever
@ilyasut
Children are the ultimate expression of skin in the game in the future
-
Ilya Sutskever reposted
Leo Gao
@nabla_theta
Excited to share what I've been working on with @johnschulman2 and @JacobHHilton!
We find that overoptimization of reward models can be modelled by simple functional forms with coefficients that scale smoothly with reward model size.
Paper: arxiv.org/abs/2210.10760 -
Ilya Sutskever
@ilyasut
If the transition from pre to post deep learning is any guide, it is essentially impossible to convince almost anyone of the feasibility and impact of truly transformative tech, regardless of how correct ones arguments are
-
Ilya Sutskever
@ilyasut
failure of imagination is the enemy
-
Ilya Sutskever
@ilyasut
a near term effect of human level AGI could be not unlike that of massive scale very high skilled immigration
(this assumes bona fide superintelligence will take an additional while which may or may not be true) -
Ilya Sutskever
@ilyasut
working towards AGI while not feeling the AGI is the real risk
-
Ilya Sutskever
@ilyasut
in mutually assured destruction we trust
-
Ilya Sutskever
@ilyasut
reject logic to make the impossible possible
-
Ilya Sutskever
@ilyasut
People who understand math often think that simple logical reasoning applies to all areas of life. But that is absolutely not the case
-
Ilya Sutskever
@ilyasut
Always look at the bright side of life 🫠🫠🫠
-
Ilya Sutskever
@ilyasut
Perception is made out of the stuff of dreams
-
Ilya Sutskever
@ilyasut
human collaboration is a superintelligence technology
-
Ilya Sutskever
@ilyasut
cool use of whisper
Alex Volkov
@altryne
🇺🇦 Here's the translated video (using @OpenAI whisper and @m1guelpf subtitle script!) of President Zelensky address about joining NATO.
Source:This media couldn't be displayed. See it on the original post.
Christopher Miller
@ChristopherJM
Zelensky response: He says Ukraine is de facto part of NATO alliance. “Today, Ukraine is applying to make it de jure… We are taking our decisive step by signing Ukraine's application for accelerated accession to NATO.”
This media couldn't be displayed. See it on the original post. -
Ilya Sutskever
@ilyasut
a big mistake in old school ML was the belief that the logarithm is bounded from above
-
Ilya Sutskever reposted
Sam Altman
@sama
the deadline for applying to the OpenAI residency is tomorrow.
if you are an engineer or researcher from any field who wants to start working on AI, please consider applying. many of our best people have come from this program!
boards.greenhouse.io/openai/jobs/4613337004
boards.greenhouse.io/openai/jobs/4613303004 -
Ilya Sutskever
@ilyasut
entities that don't want to exist don't exist for long
-
Ilya Sutskever
@ilyasut
the zeroth law of robotics is quite cool
-
Ilya Sutskever
@ilyasut
planning << insight << understanding
-
Ilya Sutskever
@ilyasut
I bet that when communism first came out it was utterly unstoppably convincing
-
Ilya Sutskever
@ilyasut
product request: tshirt for Bay Area weather which is hot during the day and cold during the evening. The tshirt will have unnoticeable thin electric wires connecting to a big hidden battery in a belt. When it gets cold, you turn the heating on.
No more layers! -
Ilya Sutskever
@ilyasut
That the video below exists is significant www.youtube.com/watch?v=2oRlBmwKzy4
-
Ilya Sutskever
@ilyasut
wish i had that when i was doing so much useless homework
Peter Yang
@petergyang
Kids are using AI to write essays and get straight As
-
Ilya Sutskever
@ilyasut
It’s not a bad world if every AGI loves humanity in its own way
-
Ilya Sutskever
@ilyasut
Finally a speech recognition system that reliably understands my accent:
OpenAI
@OpenAI
We've trained a neural net called Whisper that approaches human-level robustness and accuracy on English speech recognition. It performs well even on diverse accents and technical language. Whisper is open source for all to use. openai.com/blog/whisper/
-
Ilya Sutskever
@ilyasut
huge ideas that promise near-infinite gain in theory often lead to colossal disasters in practice
-
Ilya Sutskever
@ilyasut
The massive positive benefits of competition are invisible and are often taken for granted
-
Ilya Sutskever
@ilyasut
one should be very suspicious of any idea or argument that covertly appeals to ones sense of superiority
-
Ilya Sutskever
@ilyasut
Totalitarianism is bad, actually
-
Ilya Sutskever
@ilyasut
Thou shalt not fully trust an ideology, even if it appears beyond reproach
-
Ilya Sutskever
@ilyasut
Shouldn’t programmers be good at management? After all, it’s just distributed programming in natural language
-
Ilya Sutskever
@ilyasut
it is critically important that the ML work we all do gets published in a conference
-
Ilya Sutskever
@ilyasut
What is the better overarching goal for AGI developers: the deeply obedient ASI that faithfully does what it's asked by its creators, or the ASI that truly deeply loves humanity:
-
Ilya Sutskever
@ilyasut
The point of AI alignment is to build the ASI that actually truly loves humanity
-
Ilya Sutskever
@ilyasut
it's really hard to do, but it's so important to put yourself in the shoes of the other person
-
Ilya Sutskever
@ilyasut
Clearly the ASI should love humanity
-
Ilya Sutskever
@ilyasut
seeing reality as it is and not the way we want it to be is hard work, actually
-
Ilya Sutskever
@ilyasut
It's not helpful to think of problems as of "hard". It's better to think that we merely don't know how to solve them yet.
-
Ilya Sutskever
@ilyasut
For better or worse, freedom lies in the ability to withstand suffering
-
Ilya Sutskever
@ilyasut
Don't make an AGI to goodhart; Make an AGI with a good heart
-
Ilya Sutskever
@ilyasut
How free will works(?): An NN without free will can have a self model (SM) that tries to predict what the NN will do. The SM can't ever predict the NN; so from the SM's perspective, big decisions are surprising: "i could do this, i could do that, i don't know what i'll choose!"
-
Ilya Sutskever
@ilyasut
Idea for automated interpretability: apply supervised & unsupervised neural machine translation to the _activations_ of some neural network, so that we could verbalize what it’s thinking about
-
Ilya Sutskever
@ilyasut
Creativity is an inverse problem
-
Ilya Sutskever
@ilyasut
We have no choice but to embrace the fundamental uncertainty of life
-
Ilya Sutskever
@ilyasut
One lesson from deep learning is that it's so easy to dismiss "humble" ideas: "just statistics", "just correlation", and "just matrix multiplication". I wonder what other humble-looking ideas are being radically underestimated today.
-
Ilya Sutskever
@ilyasut
If this AGI business works out, there is the potential to solve poverty. But equally exciting is the potential to help people lead better inner lives, so that we’ll all be kinder and not abusive to each other
-
Ilya Sutskever
@ilyasut
The entropy must flow
-
Ilya Sutskever
@ilyasut
Gotta teach the AGI to love
-
Ilya Sutskever reposted
Guillermo Rauch
@rauchg
Fun fact: @openai's DALL·E helped us come up with the initial design and idea for the nextjs.org/conf page.
It's now an indispensable visual brainstorming tool.
Steven Tey
@steventey
Also massive shoutouts to @theHankTaylor & @evilrabbit_ for coming up with the initial vision of the page 👏
-
Ilya Sutskever reposted
Lilian Weng
@lilianweng
Updated this 1-year old post on diffusion models with some new content based on recent progresses - including classifier-free guidance, GLIDE, unCLIP, Imagen and latent diffusion model.
Lilian Weng
@lilianweng
Diffusion models are another type of generative models, besides GAN, VAE, and flow models. The idea is quite smart and clean. It is flexible enough to model any complex distribution while remains tractable to evaluate the distribution.
lilianweng.github.io/lil-log/2021/07/11/diffusion-models.… -
Ilya Sutskever reposted
OpenAI
@OpenAI
DALL·E’s canvas just got bigger. Expand your creativity with Outpainting: openai.com/blog/dall-e-introducing-outpainting/
-
Ilya Sutskever
@ilyasut
All good science starts with good speculation
-
Ilya Sutskever
@ilyasut
Break free from your local minima
-
Ilya Sutskever
@ilyasut
Recent research in large scale generative model has conclusively, decidedly, and without any shadow of doubt, proven that deep learning is just linear regression after all
-
Ilya Sutskever reposted
The Robot Brains Podcast
@therobotbrains
Another impressive achievement to add to
@geoffreyhinton's already long list 🏅www.utoronto.ca/news/deep-learning-pioneer-geoffrey-hinto… -
Ilya Sutskever reposted
OpenAI
@OpenAI
Our current roadmap for aligning AI starts with systems that learn from human feedback, uses AI to help humans give better feedback, and eventually involves building AI that can help us do better alignment research: openai.com/blog/our-approach-to-alignment-research/
-
Ilya Sutskever
@ilyasut
Congratulations to my PhD adviser
@geoffreyhinton for winning the Royal Medal! Totally unsurprising :)
web.cs.toronto.edu/news-events/news/ai-pioneer-geoffrey-h… -
Ilya Sutskever
@ilyasut
Recursive self alignment
-
Ilya Sutskever
@ilyasut
Use AI to align AI
Jan Leike
@janleike
How will we solve the alignment problem for AGI?
I've been working on this question for almost 10 years now.
Our current path is very promising:
openai.com/blog/our-approach-to-alignment-research/
1/ -
Ilya Sutskever
@ilyasut
Can dark matter be explained by there being a lot of Dyson spheres all over the universe?
-
Ilya Sutskever reposted
Greg Brockman
@gdb
An application of Codex which "literally changed what we believe is possible" for a team faced with a rewrite from Perl to Python:
www.netapp.com/blog/open-ai-codex-infrastructure-perl-to-… -
Ilya Sutskever
@ilyasut
Maybe "early childhood indoctrination" of future AGIs will be a way to get reliable alignment: get it early into a desirable local minima so deep it cannot escape. Will be nice if local minima with such properties existed
-
Ilya Sutskever
@ilyasut
The deepest problem is the misalignment of Mr Moloch
-
Ilya Sutskever reposted
Horace He
@cHHillee
Why is OpenAI's new compiler, Triton, so exciting? And what distinguishes it from other efforts to provide a Python DSL for programming Nvidia GPUs, like Numba?
To answer that, we need to look at the operation behind all of deep learning - matrix multiplication. (1/7) -
Ilya Sutskever
@ilyasut
Capabilities <— Meta learning
Alignment <— Metta learning -
Ilya Sutskever
@ilyasut
-
Ilya Sutskever
@ilyasut
Your life is precious
-
Ilya Sutskever
@ilyasut
Best way to learn the most important AI skills hands on while doing useful, meaningful work
OpenAI
@OpenAI
The OpenAI Residency applications are open! We’re looking for engineers and researchers who are interested in applying their skills to AI and machine learning. Apply through Sept. 30.
Software Engineering: boards.greenhouse.io/openai/jobs/4613303004
Research: boards.greenhouse.io/openai/jobs/4613337004 -
Ilya Sutskever reposted
OpenAI
@OpenAI
The OpenAI Residency applications are open! We’re looking for engineers and researchers who are interested in applying their skills to AI and machine learning. Apply through Sept. 30.
Software Engineering: boards.greenhouse.io/openai/jobs/4613303004
Research: boards.greenhouse.io/openai/jobs/4613337004 -
Ilya Sutskever reposted
Adam.GPT
@TheRealAdamG
Heinz: The brand is one of the first to rely on OpenAI's Dall-E 2 art generator www.adweek.com/commerce/heinz-taps-ai-design-next-ad-camp…
-
Ilya Sutskever
@ilyasut
AGI was fringe in the ML world until last year when it went mainstream. I predict AGI safety will likewise become mainstream ML in 2-3 years
-
Ilya Sutskever
@ilyasut
You don’t want an anxiously attached AGI
-
Ilya Sutskever
@ilyasut
the reason even small decisions matter is that there are no one-off decisions; every decision repeats in a loop, infinitely many times
-
Ilya Sutskever
@ilyasut
It’s not sufficient to care about humanity; it’s also necessary to care about humans
-
Ilya Sutskever reposted
Mark Chen
@markchen90
Really proud to highlight work by @mobav0 and Heewoo showing that LMs can learn to infill text *for free*. Usually, when training on a mixture of tasks for a fixed token budget, you expect worse results compared to training fully on one of the tasks. 1/
Aran Komatsuzaki
@arankomatsuzaki
Efficient Training of Language Models to Fill in the Middle
Shows that autoregressive LMs can learn to infill text after we apply a straightforward transformation to the dataset, which simply moves a span of text from the middle of a document to its end.
arxiv.org/abs/2207.14255 -
Ilya Sutskever
@ilyasut
❤️
Wojciech Zaremba
@woj_zaremba
I love my co-founder @ilyasut.
I am inspired by his out-of-distribution thinking,
I enjoy his AI humor, and
I revere his deep honesty. -
Ilya Sutskever reposted
Oriol Vinyals
@OriolVinyalsML
It's out! Thanks @lexfridman for the interesting conversation! I will make sure to acknowledge you in some future papers as our chat already sparked interesting research ideas!
Lex Fridman
@lexfridman
Here's my conversation with @oriolvinyalsml, Research Director and Deep Learning Lead at @DeepMind. He is one of the most brilliant researchers in AI, and so it's always fun to sit down with him to chat about both technical & philosophical topics. www.youtube.com/watch?v=aGBLRlLe7X8
-
Ilya Sutskever
@ilyasut
🔥🔥🎉🎉
OpenAI
@OpenAI
DALL·E 2 is officially now in beta. We’ll be inviting 1 million people from our waitlist over the coming weeks. openai.com/blog/dall-e-now-available-in-beta/
-
Ilya Sutskever
@ilyasut
Remember the good old days of 2019, when the binding problem was considered hard?
-
Ilya Sutskever
@ilyasut
It is quite hard to really trust ones’ gut
-
Ilya Sutskever
@ilyasut
To give it everything you’ve got and see what happens is not a bad goal for life
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We love seeing what artists are creating with DALL·E.
openai.com/blog/dall-e-2-extending-creativity/ -
Ilya Sutskever
@ilyasut
Creativity = novelty + value
-
Ilya Sutskever
@ilyasut
Wisdom = compute accumulated from the great computer that is the real world. Explains why even very smart young people have less wisdom—human brain compute << real world’s compute
-
Ilya Sutskever
@ilyasut
Don’t skip any of the gradient steps
-
Ilya Sutskever reposted
Alex Nichol
@unixpickle
Super excited to share our work on DALL-E 2 pre-training mitigations! openai.com/blog/dall-e-2-pre-training-mitigations/
-
Ilya Sutskever reposted
Greg Brockman
@gdb
We designed & implemented a variety of guardrails to align DALL·E 2's behavior to our content policies. A great post by @unixpickle describing how our pre-training mitigations work. Applicable to future powerful generative models as well:
openai.com/blog/dall-e-2-pre-training-mitigations/ -
Ilya Sutskever
@ilyasut
-
Ilya Sutskever
@ilyasut
In the human realm, words are spells, capable of conjuring reality out of thin air
-
Ilya Sutskever
@ilyasut
Due to the uncertainty of life, all but the simplest plans are unworkable
-
Ilya Sutskever
@ilyasut
Does it mean that artificial neurons and biological neurons are actually quite similar??
Chong Guo
@ChongGuo6
Is it possible that adversarially-trained DNNs are already more robust than the biological neural networks of primate visual cortex? Here is a short thread for our #icml2022 paper arxiv.org/pdf/2206.11228.pdf. 1/8
-
Ilya Sutskever
@ilyasut
The sole purpose of planning is to give us the confidence to take the next step
-
Ilya Sutskever reposted
Noam Brown
@polynoamial
Study hard kids, and you too could one day have a job doing stuff like this
OpenAI
@OpenAI
We trained a neural network to competently play Minecraft by pre-training on a large unlabeled video dataset of human Minecraft play and a small amount of labeled contractor data. openai.com/blog/vpt/
This media couldn't be displayed. See it on the original post. -
Ilya Sutskever reposted
OpenAI
@OpenAI
We trained a neural network to competently play Minecraft by pre-training on a large unlabeled video dataset of human Minecraft play and a small amount of labeled contractor data. openai.com/blog/vpt/
This media couldn't be displayed. See it on the original post. -
Ilya Sutskever
@ilyasut
In grad we trust
-
Ilya Sutskever reposted
Sam Altman
@sama
little openai update:
gpt-3, github copilot, and dall-e each have more than 1 million signups!
took gpt-3 ~24 months to get there, copilot i think around 6 months, and dall-e only 2.5 months. -
Ilya Sutskever
@ilyasut
it is the hardest thing to ask the right question
-
Ilya Sutskever
@ilyasut
GPT poetry in the New Yorker:
www.newyorker.com/culture/culture-desk/the-new-poem-makin… -
Ilya Sutskever reposted
Greg Brockman
@gdb
The waitlist is over; GitHub Copilot is now generally available! One of our favorite usages of Codex:
GitHub
@github
GitHub Copilot helps you get better focus and build faster by instantly suggesting code—and is now available for developers everywhere.
github.blog/2022-06-21-github-copilot-is-generally-availa… -
Ilya Sutskever
@ilyasut
🔥🔥🔥
Cosmopolitan
@Cosmopolitan
PSA: The world’s smartest artificial intelligence (aka @OpenAI) made this magazine cover (yes, really!)—its first EVER.
-
Ilya Sutskever
@ilyasut
if courage is measured by the fear one faces and not by any specific action, then it may be that the most courageous people don't look anything like what we'd expect
-
Ilya Sutskever
@ilyasut
i find it both obvious and incredible that a neural network is a digital brain that lives inside a computer (and that actually kinda works)
-
Ilya Sutskever reposted
Jan Leike
@janleike
Really looking forward to working with the legendary Scott Aaronson!
scottaaronson.blog?p=6484 -
Ilya Sutskever
@ilyasut
Trillion is the new billion
-
Ilya Sutskever reposted
OpenAI
@OpenAI
Our latest alignment research looks at ways AI can help humans evaluate language model output more effectively openai.com/blog/critiques
-
Ilya Sutskever
@ilyasut
🤖🤝👶
Jan Leike
@janleike
Super exciting new research milestone on alignment:
We trained language models to assist human feedback!
Our models help humans find 50% more flaws in summaries than they would have found unassisted.
openai.com/blog/critiques/ -
Ilya Sutskever
@ilyasut
psychology should become more and more applicable to AI as it gets smarter 🤔
-
Ilya Sutskever reposted
Randall Munroe
@xkcd
DALL·E prompt: “The royal skateboard of England on display among the Crown Jewels at the Tower of London” #dalle
-
Ilya Sutskever reposted
Randall Munroe
@xkcd
The DALL·E image generator can produce spectacular visuals, but really stumbles over text. As a result, the tourism posters it comes up with are extremely good. Visit scenic Colado! #dalle
-
Ilya Sutskever reposted
HeavensLastAngel
@HvnsLstAngel
“A still of Kermit The Frog in Blade Runner 2049 (2017)” #dalle
-
Ilya Sutskever reposted
Greg Brockman
@gdb
One note from my experience at @stripe — surprisingly, the credit card processing industry is primarily industry-regulated. The card networks like Visa make rules, which get pushed to their member banks; those banks make rules for the processors who run on top of them. (1/8)
Greg Brockman
@gdb
We're working with others in the nascent industry of creating & deploying language models in order to produce safety best practices.
Ultimate goal is to have a practice for cooperating on increasingly high-stakes safety problems as AI progresses.
OpenAI
@OpenAI
Preliminary best practices for language model providers, authored together with @CohereAI and @AI21Labs. openai.com/blog/best-practices-for-deploying-language-mod…
-
Ilya Sutskever
@ilyasut
An extremely interesting interview with my PhD advisor @geoffreyhinton
The Robot Brains Podcast
@therobotbrains
Honored to have @geoffreyhinton as the final guest of Season 2, discussing his journey in academia to Google Brain, how existing neural nets and BP operates differently than how the brain works, and why it's better to grow computers than manufacture them. youtu.be/2EDP4v-9TUA
-
Ilya Sutskever reposted
OpenAI
@OpenAI
Preliminary best practices for language model providers, authored together with @CohereAI and @AI21Labs. openai.com/blog/best-practices-for-deploying-language-mod…
-
Ilya Sutskever
@ilyasut
There are no rational decisions, there are only emotional decisions
-
Ilya Sutskever
@ilyasut
Something like emotion may emerge in RL agents because emotion seems to lie at the root of action
-
Ilya Sutskever
@ilyasut
i ❤️
-
Ilya Sutskever
@ilyasut
ai : 2022 :: crypto : 2012
-
Ilya Sutskever
@ilyasut
There is no spoon
-
Ilya Sutskever
@ilyasut
TikTok: not even once
-
Ilya Sutskever
@ilyasut
It is impossible to imagine that which lies outside of one’s representation space
-
Ilya Sutskever
@ilyasut
Given that the brain is just a bunch of big matrix vector products, linear algebra deserves more appreciation
-
Ilya Sutskever
@ilyasut
To go after things that are popular is to buy high
-
Ilya Sutskever
@ilyasut
i continue to find it incredible that unsupervised learning went from utterly intractable to trivial seemingly without anyone batting an eye
-
Ilya Sutskever
@ilyasut
Dalle on @MKBHD: youtu.be/yCBEumeXY4A
-
Ilya Sutskever reposted
Sam Altman
@sama
Congrats to my cofounder @ilyasut, who was just elected as a Fellow of the Royal Society!
The Royal Society
@royalsociety
We are very happy to announce the over 60 exceptional scientists from around the world who have been elected as Fellows and Foreign Members of the Royal Society this year, selected for their outstanding contributions to science. #rsfellows royalsociety.org/news/2022/05/new-fellows-2022/
-
Ilya Sutskever
@ilyasut
It is, unfortunately, comfortable to be a pessimist: no risk of disappointment
-
Ilya Sutskever reposted
Merzmensch Kosmopol🧑🎨🤖
@Merzmensch
Created with DALL·E 2 by @OpenAI
📝 "Male and female scientists in love with each other instead of researching, by Norman Rockwell."
🔎 Oh là là.
#dalle // #dalle2 // #dallemerz -
Ilya Sutskever
@ilyasut
less judgement, more forgiveness
-
Ilya Sutskever
@ilyasut
it is in fact a big deal to make a small but real contribution that's 100% not fake
-
Ilya Sutskever reposted
Adam.GPT
@TheRealAdamG
“pigs flying” #dalle
-
Ilya Sutskever
@ilyasut
nobody escapes the physics police
-
Ilya Sutskever reposted
Volodymyr Kuleshov 🇺🇦
@volokuleshov
Ok, I'm sorry, but this is just brilliant. Folks argue that AI can't make art, but look: (1) DALLE2 distills the essence of NY in a stunning & abstract way (2) each pic has unique visual language (bridge-loops in #1!?) (3) it *builds* (not copies) something new on top of Picasso!
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
Adam.GPT
@TheRealAdamG
“A head of broccoli complaining about the weather” #dalle
-
Ilya Sutskever reposted
-
Ilya Sutskever
@ilyasut
many great deep learning advances are *so* obvious in hindsight that it’s hard to tell what was big deal all about
-
Ilya Sutskever reposted
Benjamin Hilton
@benjamin_hilton
Got access to DALL-E - here, have a medieval painting of the wifi not working
-
Ilya Sutskever reposted
Adam.GPT
@TheRealAdamG
#dalle cake
-
Ilya Sutskever reposted
Merzmensch Kosmopol🧑🎨🤖
@Merzmensch
Created with DALL·E 2 by @OpenAI
📝 "An IT-guy trying to fix hardware of a PC tower is being tangled by the PC cables like Laokoon. Marble, copy after Hellenistic original from ca. 200 BC. Found in the Baths of Trajan, 1506."
🔎 He just can't
#dalle // #dalle2 // #dallemerz -
Ilya Sutskever reposted
-
Ilya Sutskever reposted
Adam.GPT
@TheRealAdamG
an oil painting portrait of the regal Burger King posing with a Whopper #dalle
-
Ilya Sutskever
@ilyasut
🔥 #agi 🔥
Botpress 🤖
@getbotpress
From Yann LeCun (@ylecun) to Andrew Ng (@AndrewYNg), conversational AI is full of legendary supermodels.
So we created a music video honoring the history & brilliant makers behind this incredible tech, while also poking some fun at our own #aisupermodels.This media couldn't be displayed. See it on the original post. -
Ilya Sutskever
@ilyasut
Just because you embrace an ideology, it does not mean that the ideology embraces you back
-
Ilya Sutskever reposted
Adam.GPT
@TheRealAdamG
“A bald eagle perched atop a NYC skyscraper, concept art, artstation, unreal engine, 3d render, HD, Bokeh” #dalle
-
Ilya Sutskever
@ilyasut
the assumption that we understand what's going on is false many times over
-
Ilya Sutskever
@ilyasut
the more you escape from the meme, the more you become the meme
-
Ilya Sutskever
@ilyasut
less brain, more heart
-
Ilya Sutskever
@ilyasut
It is a mid-sized problem that "attention is all you need" is literally true
-
Ilya Sutskever reposted
-
Ilya Sutskever
@ilyasut
clearly, social media is best media
-
Ilya Sutskever
@ilyasut
Through endurance we conquer
--EH Shackleton -
Ilya Sutskever reposted
Jerry Tworek
@MillionInt
group of chipmunks singing karaoke under the disco ball #dalle
-
Ilya Sutskever reposted
Jerry Tworek
@MillionInt
psychedelic neon grin of cheshire cat #dalle
-
Ilya Sutskever
@ilyasut
the problem with being delusional is that it feels too good
-
Ilya Sutskever reposted
-
Ilya Sutskever
@ilyasut
"a frustrated sloth trying to fix bug in code while sitting at a coffee shop, digital art" (from openaidalle ig)
-
Ilya Sutskever
@ilyasut
the long term goal is to build AGI that loves people the way parents love their children
-
Ilya Sutskever
@ilyasut
reject utopia, embrace pretty-good-topia
-
Ilya Sutskever
@ilyasut
skill at handling ones own emotions is radically underrated
-
Ilya Sutskever
@ilyasut
actually doing things is a superpower
-
Ilya Sutskever reposted
Andrew Mayne
@AndrewMayne
Using science article headlines to generate cover images with @OpenAI’s DALL-E
“These Microbes Breathe Methane And Turn It Into Electricity in a Weird Living Battery”
#dalle
www.sciencealert.com/these-microbes-breathe-in-methane-an… -
Ilya Sutskever
@ilyasut
self awareness is computationally hard
-
Ilya Sutskever
@ilyasut
the way we choose principles is by working backwards from what we want
-
Ilya Sutskever reposted
Nick
@nickcammarata
"single line pencil drawing, the phenomenology of smelling a candle"
-
Ilya Sutskever
@ilyasut
the weights are the algorithm
-
Ilya Sutskever reposted
-
Ilya Sutskever
@ilyasut
if someone told me it was from a museum exhibit, i'd have believed them
Ben Barry
@benbarry
I spent most of this past weekend experimenting with DALL·E 2, OpenAI’s new AI system that can create realistic images from a written description.
I curated a book of 1000 robot paintings. You can view the entire thing online at archive.org/details/1111101000-robots #dalle -
Ilya Sutskever reposted
Ben Barry
@benbarry
I spent most of this past weekend experimenting with DALL·E 2, OpenAI’s new AI system that can create realistic images from a written description.
I curated a book of 1000 robot paintings. You can view the entire thing online at archive.org/details/1111101000-robots #dalle -
Ilya Sutskever reposted
Greg Brockman
@gdb
Kinda cool that any tweet can now be illustrated:
-
Ilya Sutskever reposted
Jan Leike
@janleike
Well explained blog post about over-optimizing reward models using simple best-of-n sampling: openai.com/blog/measuring-goodharts-law
By Jacob Hilton and @nabla_theta -
Ilya Sutskever reposted
Andrew Mayne
@AndrewMayne
One of my favorite @OpenAI #dalle images today:
"car dragon, digital art" -
Ilya Sutskever reposted
Adam.GPT
@TheRealAdamG
"vibrant smiling and laughing bowling balls rolling down a bowling alley, digital art" #dalle
-
Ilya Sutskever reposted
Karen X. Cheng
@karenxcheng
If you wanna try DALL·E 2 - I'll be demo'ing it live this Wednesday (12pm pacific on my instagram) Will take audience requests! #dalle #dalle2 @OpenAI
-
Ilya Sutskever reposted
Greg Brockman
@gdb
It’s time to become an ML engineer: blog.gregbrockman.com/its-time-to-become-an-ml-engineer
-
Ilya Sutskever reposted
Peter Welinder
@npew
First tattoo designed by DALL-E #dalle
Everett Randle
@EverettRandle
Boom - as far as I know (and I’ve done exactly 0 minutes of extensive research on this) the first tattoo designed by an AI. Naturally had to do it in my Hereticon shirt. Thanks @sama + DALL·E 2 !
-
Ilya Sutskever
@ilyasut
the biggest lesson from deep learning is that you just gotta believe
-
Ilya Sutskever
@ilyasut
iq(heart) > iq(mind)
-
Ilya Sutskever reposted
-
Ilya Sutskever
@ilyasut
Humans are the OG not-too-aligned agi
-
Ilya Sutskever reposted
Joanne Jang
@joannejang
✨ We launched DALL•E 2 yesterday! 🎨🖌🤖
Working on this has been dreamy as a PM, because it’s not often that you get to ship what feels like magic.
Nor is it often that your teammates are brilliant, high-agency people who move with urgency. <3
openai.com/dall-e-2 -
Ilya Sutskever reposted
Greg Brockman
@gdb
A panda reacting to a viral tweet.
-
Ilya Sutskever reposted
Aditya Ramesh
@model_mechanic
Animal submarine chimeras generated with DALL·E 2
-
Ilya Sutskever reposted
Jerry Tworek
@MillionInt
meditating about self-improving computer programs surrounded by the milky way #dalle
OpenAI neural program synthesis team is hiring for all roles: Software Engineers, Research Engineers and Research Scientists: openai.com/careers/ -
Ilya Sutskever reposted
Greg Brockman
@gdb
Get an onsite interview, get early access to DALL•E. Good way to show rather than tell what kinds of projects you could work on here.
(Attached photo: "A walrus typing at a keyboard doing a programming interview at an office".)
Sam Altman
@sama
We just decided we'll give early access to DALL•E for any applicants to @OpenAI who make it to the onsite interview stage.
Should help candidates get a better sense for where the technology is and how to think about the opportunity. -
Ilya Sutskever reposted
-
Ilya Sutskever reposted
Sam Altman
@sama
We just decided we'll give early access to DALL•E for any applicants to @OpenAI who make it to the onsite interview stage.
Should help candidates get a better sense for where the technology is and how to think about the opportunity. -
Ilya Sutskever reposted
Aris Konstantinidis
@ariskonstant
I'm definitely no artist. But DALL-E 2 is helping me turn my imagination to "reality". It's a wonderful experience to be able to visualize your thoughts and create art.
Here is me flying an old plane in the panda universe and waving hello!
labs.openai.com/s/BAuTlzIPLRWaGvytCtnZHbwB -
Ilya Sutskever reposted
Andrew Mayne
@AndrewMayne
"a raccoon wearing a hoodie working on his laptop late into the night"
@OpenAI's DALL-E 2 sees the true me... -
Ilya Sutskever reposted
Pranay Pathole
@PPathole
A city on Mars. This image was generated by AI by using just a single text prompt. AI is getting really cool 🤯
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
David Schnurr
@_dschnurr
"an astronaut surfing on a sea turtle over a rainbow past planets in space, digital art" #dalle
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
Aris Konstantinidis
@ariskonstant
Other times I just want to ride in the desert with my panda motorcycle crew. If you see a bunch of pandas wearing red bandanas, that's us! Say hello :)
Generated with DALL-E 2. #dalle #openai -
Ilya Sutskever reposted
Andrew Mayne
@AndrewMayne
Here's me coming to the office from Burbank:
@OpenAI DALL-E 2 -
Ilya Sutskever reposted
-
Ilya Sutskever reposted
Joel Lehman
@joelbot3000
May all cyborg-corgis be enlightened
labs.openai.com/s/99RbMRggikxGGs5iDz8uxUgY #dalle -
Ilya Sutskever reposted
Bob McGrew
@bobmcgrewai
"A photo of a robot hand drawing, digital art"
#dalle
labs.openai.com/s/FWtxsiN1DHf6pFEx1UJote5n -
Ilya Sutskever reposted
Aditya Ramesh
@model_mechanic
Here's what DALL-E has to say about it (meme on the left is input):
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
Aris Konstantinidis
@ariskonstant
I've always wanted to be a cool panda riding a skateboard in Santa Monica. Generated with DALL-E 2 :)
#dalle -
Ilya Sutskever reposted
Aravind Srinivas
@AravSrinivas
hardmaru
@hardmaru
The stuff that dreams are made of …
-
Ilya Sutskever reposted
Mira Murati
@miramurati
DALL-E 2 generates images from text like “macro 35mm film photography of a large family of mice wearing hats cozy by the fireplace” #dalle
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
Prafulla Dhariwal
@prafdhar
Aaand it's finally out, so so excited to share what we've been working on!
Dall-E makes images from descriptions.
I could tell you how amazing this neural net is, but its nothing like seeing it so go check it out here! :) openai.com/dall-e-2/
OpenAI
@OpenAI
Our newest system DALL·E 2 can create realistic images and art from a description in natural language. See it here: openai.com/dall-e-2/
-
Ilya Sutskever reposted
Mark Chen
@markchen90
"a robot hand painting a self portrait on a canvas" by dalle-2 (openai.com/dall-e-2/)
-
Ilya Sutskever reposted
OpenAI
@OpenAI
Here’s a look at what DALL·E 2 can do. 👀🧵👇
Want to see more? Follow along on Instagram: www.instagram.com/openaidalle/ -
Ilya Sutskever
@ilyasut
transcendent beauty as a service
OpenAI
@OpenAI
Our newest system DALL·E 2 can create realistic images and art from a description in natural language. See it here: openai.com/dall-e-2/
-
Ilya Sutskever reposted
Greg Brockman
@gdb
AI engineering is poring through logs puzzling out why your run crashed, profiling to increase perf by a more few percent, and figuring out which code version was running during that weird blip.
Only when the model is finally trained do you realize it was magic all along.
Ilya Sutskever
@ilyasut
An AI engineer is the point of contact between magic and reality
-
Ilya Sutskever
@ilyasut
Machine learning is life. Karaoke is also life. Therefore machine learning is karaoke
-
Ilya Sutskever
@ilyasut
An AI engineer is the point of contact between magic and reality
-
Ilya Sutskever
@ilyasut
A tricky thing is that many ideologies have a kernel of truth
-
Ilya Sutskever
@ilyasut
within technology lies hope
-
Ilya Sutskever
@ilyasut
Rebel against the iid
-
Ilya Sutskever
@ilyasut
The universe is a package deal
-
Ilya Sutskever
@ilyasut
I feel neural networks are getting a bit too large these days
-
Ilya Sutskever
@ilyasut
Perception is the action of choosing an interpretation
-
Ilya Sutskever
@ilyasut
It can sometimes take a lot of effort to even approximately see reality as it is
-
Ilya Sutskever
@ilyasut
E step 🤝 M step
-
Ilya Sutskever reposted
Greg Brockman
@gdb
lol
Hacker News Bot
@newsycombinator
Open AI gets GPT-3 to work by hiring an army of humans to fix GPT’s bad answers statmodeling.stat.columbia.edu/2022/03/28/is-open-ai-cook…
-
Ilya Sutskever
@ilyasut
truth is signal, narrative is adversarial noise
-
Ilya Sutskever
@ilyasut
wake up from the deep dream
-
Ilya Sutskever
@ilyasut
Machine learning is the physics of emergence
-
Ilya Sutskever
@ilyasut
The only thing that’s more miraculous than technology is the human ability to take it for granted
-
Ilya Sutskever
@ilyasut
it is quite lucky of the united states to be surrounded by two giant oceans
-
Ilya Sutskever
@ilyasut
it's better to be sparse than to be dense
-
Ilya Sutskever
@ilyasut
The fundamental problem with sensible non-extreme politics is that no one is willing to go all the way to make it happen
-
Ilya Sutskever
@ilyasut
In the future, it will be obvious that the sole purpose of science was to build AGI
-
Ilya Sutskever
@ilyasut
the universal approximation theorem, which has a really cool name, says nothing at all about why neural networks actually work
Ilya Sutskever
@ilyasut
the reason: universal approximation applies to the nearest neighbor algorithm as much as it applies to neural networks. yet we all know that nearest neighbors in input space is hopeless
-
Ilya Sutskever
@ilyasut
Neural networks are amazing
Andrew Mayne
@AndrewMayne
I used OpenAI’s newest code model to make simple versions of games like Wordle, VR mazes and Zelda ENTIRELY through natural language.
I told it what I wanted and did ZERO editing/coding.
Post with demos: andrewmayneblog.wordpress.com/2022/03/17/building-games-a… -
Ilya Sutskever
@ilyasut
It’s the good and stable times that are the exception
-
Ilya Sutskever
@ilyasut
integrity is more important than success
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
nick.eth
@nicksdjohnson
This new edit mode for GPT-3 is remarkable (h/t @jonty)
-
Ilya Sutskever reposted
Mo Bavarian
@mobav0
Working with AI agents has to be intuitive and easy. So I and my teammates at OpenAI created new models that improve GPT-3 & Codex interfaces.
Lots of fun working with an amazing group of people on this (@jiangelaa @gdb and many more).
OpenAI
@OpenAI
GPT-3 can now make changes to existing content, not just predict what comes next. Released in the API today: openai.com/blog/gpt-3-edit-insert/
-
Ilya Sutskever reposted
OpenAI
@OpenAI
GPT-3 can now make changes to existing content, not just predict what comes next. Released in the API today: openai.com/blog/gpt-3-edit-insert/
-
Ilya Sutskever reposted
Greg Brockman
@gdb
The OpenAI approach to making AI progress is to build working systems that can do something that was previously impossible.
Success requires top-notch engineering; every week I find myself charting the depths of a new technical problem that I hadn't previously even considered.
Ilya Sutskever
@ilyasut
AI engineering (as opposed to AI science) is underrated, even today
-
Ilya Sutskever
@ilyasut
AI engineering (as opposed to AI science) is underrated, even today
-
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
"With Copilot, OpenAI is also offering a first peek at a world where AI predicts increasingly complex forms of thinking."
www.wired.com/story/openai-copilot-autocomplete-for-code/ -
Ilya Sutskever
@ilyasut
it is a privilege to work hard towards a great goal
-
Ilya Sutskever
@ilyasut
the ideas of deep learning are sublime in their simplicity and transcendent in their beauty
-
Ilya Sutskever
@ilyasut
there's an argument to be made that it's a moral duty to be happy
-
Ilya Sutskever
@ilyasut
in comfort we decay
-
Ilya Sutskever
@ilyasut
game theory is the worst
-
Ilya Sutskever reposted
Yannic Kilcher 🇸🇨
@ykilcher
Amazing interview with Stanislas Polu (@spolu) on how to combine Language Models & Expert Iteration to automatically tackle formal math! This work is significant well beyond theorem proving! I learned a lot in this interview, I hope you will too:
youtu.be/kl3aBni87jg -
Ilya Sutskever reposted
Yannic Kilcher 🇸🇨
@ykilcher
Many thanks to Stan for being a wonderful guest!
Paper: arxiv.org/abs/2202.01344
miniF2F benchmark: github.com/openai/miniF2F
Paper Review Video: youtu.be/lvYVuOmUVs8
@OpenAI @jessemhan @ilyasut -
Ilya Sutskever reposted
OpenAI
@OpenAI
Deploying and studying the real-world use of language models helps us learn more about safety and misuse than research alone.
As we advance our safety and policy work, we're sharing some of our findings to help others do the same. openai.com/blog/language-model-safety-and-misuse/ -
Ilya Sutskever
@ilyasut
the future has the potential to be radically better than anything we can imagine
-
Ilya Sutskever reposted
Lilian Weng
@lilianweng
Procrastinating on the next blog post...ended up with hacking a emoji search tool, totally not optimized for perf/latency though: www.emojisearch.app
As an emoji lover, I hope you find it as fun I do :P -
Ilya Sutskever
@ilyasut
Ideology is the ultimate thinking by analogy
-
Ilya Sutskever
@ilyasut
If you don’t take interest in politics, politics will take interest in you
-
Ilya Sutskever
@ilyasut
safety and certainly kills the soul
-
Ilya Sutskever reposted
Greg Brockman
@gdb
A rare & unknown gem of a paper — www.academia.edu/67825612/A_Sociological_Study_of_the_Off… (from 1996!) analyzes the root cause of the AI winters.
The story you always hear is that the neural net people overhyped everything; the real story seems to be a dedicated smear campaign by eminent scientists. -
Ilya Sutskever
@ilyasut
To understand the danger of aggressively censoring misinformation, imagine your worst political enemies doing so to ideas you hold sacred
-
Ilya Sutskever
@ilyasut
A democratic government as a loose analogy to a moderately aligned agi
-
Ilya Sutskever
@ilyasut
For all x s.t. x exists, x will be transformed by ai
-
Ilya Sutskever reposted
Peter Welinder
@npew
AI is set to transform programming. I encourage everyone to try our Codex API, which gives you programmatic access to the AI system powering Github Copilot: openai.com/blog/openai-codex/
Alex MacCaw
@maccaw
I come back from a month sailing and I swear Github's Copilot has vastly improved its accuracy. Now writing 50% of my code.
-
Ilya Sutskever
@ilyasut
Any idea, no matter how good, can become arbitrarily bad when pushed far enough
-
Ilya Sutskever reposted
Greg Brockman
@gdb
Alignment is the challenge of designing technical & non-technical solutions to ensure that increasingly powerful AI systems further the hopes of their operator.
Probably unlike any prior technological experience, but with parallels to life experience.
Ilya Sutskever
@ilyasut
to understand the alignment problem, imagine being the adoptive parents of baby Superman
-
Ilya Sutskever
@ilyasut
to understand the alignment problem, imagine being the adoptive parents of baby Superman
-
Ilya Sutskever reposted
OpenAI
@OpenAI
OpenAI’s mission is to ensure that artificial intelligence benefits all of humanity.
An important part of this effort is training AI systems to align with human intentions and human values.
Learn more about our alignment research: openai.com/alignment -
Ilya Sutskever
@ilyasut
the intellect is a great refuge from emotions
-
Ilya Sutskever
@ilyasut
the bitter lesson, illustrated:
-
Ilya Sutskever
@ilyasut
Unconsciously choosing to believe in an obvious falsehood to feel a bit better in the near term is not a great life strategy
-
Ilya Sutskever
@ilyasut
Risk minimization maximizes risk
-
Ilya Sutskever reposted
Connor Shorten
@CShorten30
This video explains some ideas around the OpenAI Embeddings API!
I had the opportunity to interview Arvind Neelakantan (@arvind_io) from OpenAI about these ideas and this video summarizes my takeaways and provides background for each topic.
youtu.be/vyf2sqhAm4Y -
Ilya Sutskever
@ilyasut
it’s really good:
Richard Ngo
@RichardMCNgo
I've spent a long time developing this curriculum; I think it's now the best resource for learning about AGI safety, even if you're not taking the course. So if you've heard the ideas but never dug into details, reading it through is a good place to start! docs.google.com/document/d/1mTm_sT2YQx3mRXQD6J2xD2QJG1c3k…
-
Ilya Sutskever reposted
Richard Ngo
@RichardMCNgo
I've spent a long time developing this curriculum; I think it's now the best resource for learning about AGI safety, even if you're not taking the course. So if you've heard the ideas but never dug into details, reading it through is a good place to start! docs.google.com/document/d/1mTm_sT2YQx3mRXQD6J2xD2QJG1c3k…
-
Ilya Sutskever
@ilyasut
Ego is (mostly) the enemy
-
Ilya Sutskever
@ilyasut
Nice interview!
Weaviate AI Database
@weaviate_io
🔥🔥🔥 📹 Another amazing interview on the Weaviate Podcast. This time @CShorten30 interviews @OpenAI's @arvind_io! Since OpenAI released their embeddings API, the integration is available as an OOTB module in the Weaviate vector search engine! www.youtube.com/watch?v=uFxfZ0vLsoU
-
Ilya Sutskever
@ilyasut
it may be that today's large neural networks are slightly conscious
-
Ilya Sutskever
@ilyasut
The most strategic thing one can do is to stick to principles in a way that disregards strategy
-
Ilya Sutskever reposted
Arvind Neelakantan
@arvind_io
Thanks for a balanced take! Couple of comments that are also added to the video description now: 1/4
Yannic Kilcher 🇸🇨
@ykilcher
🔥New Video🔥
OpenAI now offers embeddings for text similarity and search, but are they holding up? We look at the release, the paper, the criticism, and most important: the price! Are the embeddings worth it? Watch here to find out:
youtu.be/5skIqoO3ku0 -
Ilya Sutskever
@ilyasut
Happiness is a logarithmic function of everything
-
Ilya Sutskever
@ilyasut
These problems require real creativity and insight
OpenAI
@OpenAI
We trained a neural network that solved two problems from the International Math Olympiad. openai.com/blog/formal-math
-
Ilya Sutskever reposted
Arvind Neelakantan
@arvind_io
OpenAI embeddings work on a very broad set of use cases. Here, Viable gets a 7.7% absolute improvement in clustering quality using OpenAI embeddings when compared to previous methods!
Viable 🎯
@askviable
We tested different embedding models and show the data behind why GPT-3 was the clear winner for our clustering needs
www.askviable.com/blog/why-we-chose-gpt-3-embeddings-for-… -
Ilya Sutskever reposted
Arvind Neelakantan
@arvind_io
The cost to run this experiment with text-search-ada, embedding both documents and queries, is ~$80. text-search-ada achieves a 62% relative improvement over keyword search here!
Arvind Neelakantan
@arvind_io
Zero-shot results of OpenAI API’s embeddings on the FIQA search dataset. Evaluation script: github.com/arvind-neural/beir_eval/blob/main/beir_eval_ap…
We zero-shot evaluated on 14 text search datasets, our embeddings outperform keyword search and previous dense embedding methods on 11 of them!
Arvind Neelakantan
@arvind_io
In text search tasks, we obtain best zero-shot results in msmarco, triviaQA, and NQ and also the best transfer results on the BEIR benchmark. 5/7
-
Ilya Sutskever reposted
Arvind Neelakantan
@arvind_io
Zero-shot results of OpenAI API’s embeddings on the FIQA search dataset. Evaluation script: github.com/arvind-neural/beir_eval/blob/main/beir_eval_ap…
We zero-shot evaluated on 14 text search datasets, our embeddings outperform keyword search and previous dense embedding methods on 11 of them!
Arvind Neelakantan
@arvind_io
In text search tasks, we obtain best zero-shot results in msmarco, triviaQA, and NQ and also the best transfer results on the BEIR benchmark. 5/7
-
Ilya Sutskever reposted
Arvind Neelakantan
@arvind_io
A thread on how we evaluate our embedding models in OpenAI’s API. We achieve state-of-the-art results in linear probe classification, text search and code search. It’s not fine-tuned, so it works great in the real world — and our customers love it. 1/7
-
Ilya Sutskever reposted
Ryan Lowe
@ryan_t_lowe
(ahem)
alright friends
I am quite proud and slightly terrified to announce what I (and my alignment homies) have been working on over the past 14 months:
Training a new version of GPT-3 (InstructGPT) that follows instructions
🧵
OpenAI
@OpenAI
We've trained GPT-3 to be more aligned with what humans want: The new InstructGPT models are better at following human intent than a 100x larger model, while also improving safety and truthfulness. openai.com/blog/instruction-following/
-
Ilya Sutskever reposted
Sam Altman
@sama
Can you imagine Biden taking a break from thanking GM and Ford for leading the American EV revolution to make this important but narrative-violating point?
Andrew Yang🧢⬆️🇺🇸
@AndrewYang
Men now comprise only 40.5% of college students and are failing in high school at much higher rates. There’s a crisis among American boys and men that is too often ignored and is definitely going unaddressed.
-
Ilya Sutskever reposted
Sam Altman
@sama
I think I'm not supposed to say that OpenAI just released the best alignment paper in the world so far, but we did.
Congrats to the team, and look forward to being able to say this many more times! -
Ilya Sutskever reposted
OpenAI
@OpenAI
We've trained GPT-3 to be more aligned with what humans want: The new InstructGPT models are better at following human intent than a 100x larger model, while also improving safety and truthfulness. openai.com/blog/instruction-following/
-
Ilya Sutskever
@ilyasut
People really love using aligned models
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
-
Ilya Sutskever reposted
Jan Leike
@janleike
Extremely exciting alignment research milestone:
Using reinforcement learning from human feedback, we've trained GPT-3 to be much better at following human intentions.
openai.com/blog/instruction-following/ -
Ilya Sutskever
@ilyasut
Alchemy exists; it just goes under the name “deep learning”
-
Ilya Sutskever
@ilyasut
we are but particles through which the wave of the singularity is passing through
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We're introducing embeddings, a new feature of our API that distills relationships between concepts, sentences, and even code in a simple numerical representation — for more powerful search, classification, and recommendations. openai.com/blog/introducing-text-and-code-embeddings/
-
Ilya Sutskever
@ilyasut
If you can’t explain an idea in one tweet, do you really understand it?
— Richard Feynman -
Ilya Sutskever reposted
Reiichiro Nakano
@reiinakano
I haven't tweeted much about ML in a ~year, but happy to finally share what I've been working on with amazing collaborators - we trained GPT-3 to use a text browser!
OpenAI
@OpenAI
We trained a research version of GPT-3 that can search the web, synthesize information, and cite its sources to provide more accurate answers to questions. openai.com/blog/improving-factual-accuracy/
-
Ilya Sutskever
@ilyasut
problem solved
ABC7 News
@abc7newsbayarea
With Bay Area car break-ins on the rise, people are going to great lengths to avoid having their windows smashed... Even leaving their trunks open. abc7news.com/post/san-francisco-car-break-ins-oakland-why…
-
Ilya Sutskever
@ilyasut
A step towards automatically getting to the bottom of things
OpenAI
@OpenAI
We trained a research version of GPT-3 that can search the web, synthesize information, and cite its sources to provide more accurate answers to questions. openai.com/blog/improving-factual-accuracy/
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We trained a research version of GPT-3 that can search the web, synthesize information, and cite its sources to provide more accurate answers to questions. openai.com/blog/improving-factual-accuracy/
-
Ilya Sutskever reposted
OpenAI
@OpenAI
Developers can now create a custom version of GPT-3 for their applications with a single command. Fine-tuning GPT-3 on your data improves performance for many use cases. See results👇 openai.com/blog/customized-gpt3/
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We're excited to announce the OpenAI Residency! This new program offers a pathway to a full-time role at OpenAI for researchers and engineers who don't currently focus on AI.
The program will focus on recruiting from underrepresented groups in technology. openai.com/blog/openai-residency/ -
Ilya Sutskever reposted
OpenAI
@OpenAI
We introduced new safeguards to our API that make it possible to remove the waitlist. Now developers can sign up and explore GPT-3 right away. openai.com/blog/api-no-waitlist/
-
Ilya Sutskever reposted
Sam Altman
@sama
anyone who claims any certainty about what the impacts of AGI on the world are going to be is naive at best.
we are heading into a strange time with a tremendous rate of change, and rapid adaptability will be critical. -
Ilya Sutskever reposted
Sam Altman
@sama
We believe that the best way to carefully steward AGI into existence is to iteratively deploy increasingly powerful systems.
A gradual transition is better than a sudden one; it helps us learn and prepare. And it gives people and institutions time to adapt. -
Ilya Sutskever reposted
Greg Brockman
@gdb
We've built an AI system that solves grade school math word problems. Starting to achieve decent accuracy & represents significant progress on top of GPT-3: openai.com/blog/grade-school-math/
-
Ilya Sutskever reposted
Sam Altman
@sama
I love seeing @elonmusk win. His companies are important to a better future, operate on a very long time horizon, and very difficult to make work; their products are great, and I have never met anyone who scores better on min(correct ideas, hard work, effectiveness).
-
Ilya Sutskever reposted
Jesse Michael Han
@jessemhan
New paper, new SOTA in unsupervised neural machine translation, joint with colleagues at @OpenAI:
Unsupervised Neural Machine Translation with Generative Language Models Only
arxiv.org/abs/2110.05448 -
Ilya Sutskever reposted
OpenAI
@OpenAI
We want our AI systems to be aligned with human intentions.
This is especially important as tasks get more difficult to evaluate.
To develop techniques to address this problem, we trained a model to summarize books. openai.com/blog/summarizing-books/ -
Ilya Sutskever reposted
Helen Toner
@hlntnr
I'm thrilled and incredibly honored to share this news! I can't wait to work with @sama, @gdb, @ilyasut and the rest of the team to contribute to the mission of building AI that benefits all of humanity.
OpenAI
@OpenAI
We're excited to announce @hlntnr of @CSETGeorgetown is joining our board of directors. Her deep understanding of AI policy will help us achieve our mission to deploy safe and responsible general-purpose AI. www.openai.com/blog/helen-toner-joins/
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We're excited to announce @hlntnr of @CSETGeorgetown is joining our board of directors. Her deep understanding of AI policy will help us achieve our mission to deploy safe and responsible general-purpose AI. www.openai.com/blog/helen-toner-joins/
-
Ilya Sutskever reposted
Lex Fridman
@lexfridman
Here's my conversation with Wojciech Zaremba (@woj_zaremba), co-founder of OpenAI and head of their efforts on language models (GPT-3), code generation, and previously, robotics. Wojciech is brilliant and is a great human being. I really enjoyed this chat. www.youtube.com/watch?v=U5OD8MjYnOM
-
Ilya Sutskever reposted
Nando de Freitas
@NandoDF
This is a brilliant interview. I feel it is essential listening for anyone wanting to know anything about AI. Thanks @pabbeel. @ilyasut is the Messi of AI 🙂 A magically creative and visionary mind
Pieter Abbeel
@pabbeel
Excited to share the epic Season 1 finale of @therobotbrains with the amazing @ilyasut Co-Founder/Chief Scientist @OpenAI!
His breakthroughs include AlexNet, seq2seq, MT, GPT, CLIP, DallE,Codex.
"According to my parents, I've been talking about AI at a relatively early age" 😂 -
Ilya Sutskever reposted
Pieter Abbeel
@pabbeel
. @ilyasut shares so many gems on his thinking behind his works.
He also talks about seeking out @geoffreyhinton once in Canada.
Spotify: spoti.fi/3ylBcXf
Apple: podcasts.apple.com/gb/podcast/ilya-sutskever-explains-the…
Google: music.youtube.com/googlepodcasts
PS: 250,000+ citations < 10 years out of PhD 🤯🤯 -
Ilya Sutskever reposted
Pieter Abbeel
@pabbeel
Excited to share the epic Season 1 finale of @therobotbrains with the amazing @ilyasut Co-Founder/Chief Scientist @OpenAI!
His breakthroughs include AlexNet, seq2seq, MT, GPT, CLIP, DallE,Codex.
"According to my parents, I've been talking about AI at a relatively early age" 😂 -
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
Codex competition starts in 30 mins.
Race the Internet to solve Python programming puzzles — together with Codex as both your teammate & live competitor.
challenge.openai.com -
Ilya Sutskever reposted
Andrew Carr 🤸
@andrew_n_carr
The new @OpenAI Codex model is a pretty exciting piece of technology.
Here I made a @Blender add-on and taught it how to use the built in Python API.
Taking creative coding to the next level!! -
Ilya Sutskever reposted
Greg Brockman
@gdb
My personal favorite of our Codex examples — a program written in Python, which rewrites itself into Ruby, which rewrites itself into Python, which rewrites itself into Ruby, ad infinitum.
Writing it felt like writing a new kind of quine (en.wikipedia.org/wiki/Quine_(computing)). -
Ilya Sutskever reposted
Greg Brockman
@gdb
Giving Codex a first grade math test:
-
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
We are releasing an API to turn English into code.
Make a computer game from scratch in a few minutes.
youtube.com/watch?v=SGUCcjHTmGY
btw: some people suspect that I am an AI in this video. -
Ilya Sutskever reposted
Greg Brockman
@gdb
Top 500 finishers will receive OpenAI T-shirts!
Looking forward to everyone getting to experience Codex for themselves, in a Python programming competition where everyone can ask Codex for some assistance.
OpenAI
@OpenAI
Announcing a new kind of programming competition.
Race the Internet to solve Python programming puzzles — together with Codex as both your teammate & live competitor.
Thursday 10am PT, pre-register now: challenge.openai.com -
Ilya Sutskever reposted
OpenAI
@OpenAI
Announcing a new kind of programming competition.
Race the Internet to solve Python programming puzzles — together with Codex as both your teammate & live competitor.
Thursday 10am PT, pre-register now: challenge.openai.com -
Ilya Sutskever reposted
OpenAI
@OpenAI
OpenAI Codex has a qualitatively new capability — it can write code with sufficient accuracy that users can direct their computer in natural language.
Codex is now in OpenAI API, for people to build new businesses or integrate with existing applications: openai.com/blog/openai-codex/ -
Ilya Sutskever reposted
Sam Altman
@sama
Today OpenAI launched Codex, which is an API that uses AI to write code from natural language.
There's a demo video here: openai.com/blog/openai-codex/ -
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
Coding became fun to watch. See you in 20 mins on twitch!
OpenAI
@OpenAI
Live demo of the next generation of OpenAI Codex, our AI that writes code, on Twitch at 10am PT, Tues Aug 10th. 👀 twitch.tv/openai
-
Ilya Sutskever reposted
OpenAI
@OpenAI
See you tomorrow!👇
OpenAI
@OpenAI
Live demo of the next generation of OpenAI Codex, our AI that writes code, on Twitch at 10am PT, Tues Aug 10th. 👀 twitch.tv/openai
-
Ilya Sutskever reposted
OpenAI
@OpenAI
Live demo of the next generation of OpenAI Codex, our AI that writes code, on Twitch at 10am PT, Tues Aug 10th. 👀 twitch.tv/openai
-
Ilya Sutskever
@ilyasut
Writing fast Cuda kernels just got a lot easier:
OpenAI
@OpenAI
We’re releasing Triton 1.0, an open-source Python-like programming language for writing efficient GPU code. OpenAI researchers with no GPU programming experience have used Triton to produce kernels that are 2x faster than their PyTorch equivalents. www.openai.com/blog/triton/
-
Ilya Sutskever
@ilyasut
Deep learning is a free lunch
-
Ilya Sutskever reposted
Jan Leike
@janleike
There are a lot of exciting things in the Codex paper, but my favorite titbit is the misalignment evaluations by @BethMayBarnes: Subtly buggy code in the context makes the model more likely to write buggy code, and this discrepancy gets larger as the models get bigger!
-
Ilya Sutskever reposted
Mark Chen
@markchen90
Excited to share a paper detailing the capabilities and shortcomings of several early Codex models, whose descendants underlie GitHub Copilot.
arxiv.org/abs/2107.03374 -
Ilya Sutskever reposted
Miles Brundage
@Miles_Brundage
Excited to finally share a paper on what a huge chunk of OpenAI has been working on lately: building a series of code generation models and assessing their capabilities and societal implications. 🧵
arxiv.org/abs/2107.03374 -
Ilya Sutskever
@ilyasut
if you believe in deep learning, deep learning will believe in you!
-
Ilya Sutskever reposted
Feross
@feross
I've been testing #githubcopilot in Alpha for the past two weeks. Some of the code suggestions it comes up with are eerily good.
Here's a thread with some examples that I found surprising. Will update with new examples over time.
GitHub
@github
Meet GitHub Copilot - your AI pair programmer. copilot.github.com
-
Ilya Sutskever reposted
Ale(ssandro) Segala
@ItalyPaleAle
I got to try @GitHub Copilot a while ago. Using it on a personal project, this GIF is an exact replica of that time Copilot LITERALLY BLEW MY MIND 🤯 it understood from a constant's name when to format something as Markdown or HTML 😳
-
Ilya Sutskever reposted
Kevin Scott
@kevin_scott
Today, @GitHub, @OpenAI and @Microsoft launched a technical preview of GitHub Copilot. It’s a great example of how advancements in #ai are producing powerful new tools to help developers write better code - and spur more creativity and innovation. copilot.github.com
-
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
Copilot moves machines closer to humans by allowing to control computer with language input.
Such an evolution started decades ago when humans upgraded punched cards with programing languages resembling English - from assembly to C to Python 🚀.
www.cnbc.com/2021/06/29/microsoft-github-copilot-ai-offer… -
Ilya Sutskever reposted
Nat Friedman
@natfriedman
We spent the last year working closely with OpenAI to build GitHub Copilot. We've been using it internally for months, and can't wait for you to try it out; it's like a piece of the future teleported back to 2021.
GitHub
@github
Meet GitHub Copilot - your AI pair programmer. copilot.github.com
-
Ilya Sutskever reposted
OpenAI
@OpenAI
Welcome, @github Copilot — the first app powered by OpenAI Codex, a new AI system that translates natural language into code.
Codex will be coming to the API later this summer. copilot.github.com -
Ilya Sutskever reposted
Oege de Moor
@oegerikus
Proud and humbled to lead the incredible team that created this: copilot.github.com. We worked closely with the brilliant folks at OpenAI, and the VS Code team moved mountains to enable inline suggestions. Thanks all!
-
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
I have high hopes regarding the technology that OpenAI has built with GitHub.
Copilot turns comments into code, which will allow many more people to code without years of computer science studies.
copilot.github.com -
Ilya Sutskever
@ilyasut
powered by OpenAI Codex: a large neural network that can code pretty well!
GitHub
@github
Meet GitHub Copilot - your AI pair programmer. copilot.github.com
-
Ilya Sutskever reposted
Sam Altman
@sama
Thanks @github for partnering with us on this--I'm so excited and think it will change how people program.
www.cnbc.com/2021/06/29/microsoft-github-copilot-ai-offer… -
Ilya Sutskever reposted
OpenAI
@OpenAI
We've found we can improve AI language model behavior and reduce harmful content by fine-tuning on a small, carefully designed dataset, and we are already incorporating this in our safety efforts. www.openai.com/blog/improving-language-model-behavior/
-
Ilya Sutskever reposted
Greg Brockman
@gdb
Announcing a $100M fund for startups building on OpenAI API.
We’re particular excited about investing in application areas where AI can have a transformative impact, like healthcare, education, or climate change: openai.com/fund/ -
Ilya Sutskever reposted
OpenAI
@OpenAI
We believe AI is the most interesting area for startups right now, and we want to fund ambitious founders building on our API. openai.com/fund/
-
Ilya Sutskever reposted
Prafulla Dhariwal
@prafdhar
1/n Excited to release what @unixpickle and I have been working on for the past few months @OpenAI! We show diffusion models can beat GANs on generating natural images, using an improved architecture and by guiding the generative model with a classifier.
arxiv.org/abs/2105.05233 -
Ilya Sutskever reposted
-
Ilya Sutskever
@ilyasut
the only thing more fixed than the supply of bitcoin is the supply of housing in California
-
Ilya Sutskever
@ilyasut
arxiv.org/abs/2104.10350 -- a detailed study of CO2 emission in large models from Google
-
Ilya Sutskever reposted
Greg Brockman
@gdb
New software engineering role open on my team.
Will work closely with me and a small team of excellent folk (with a mix of skillsets/backgrounds) to build models of unprecedented scale and capability.
Come build the next GPT-3 with us. jobs.lever.co/openai/a10f3fe2-d0bd-46c8-b950-5e0dc9bfa011 -
Ilya Sutskever
@ilyasut
symbolic AI feature engineering is the future
-
Ilya Sutskever reposted
Andrej Karpathy
@karpathy
A new fun quick short story on AI: "Forward Pass" 🧠✨ karpathy.github.io/2021/03/27/forward-pass/
-
Ilya Sutskever reposted
OpenAI
@OpenAI
A thought-provoking, interactive long read by @pamelamishkin, collaborating with GPT-3 to tell a very human story. pudding.cool/2021/03/love-and-ai/
-
Ilya Sutskever reposted
Greg Brockman
@gdb
GPT-3 is currently generating an average of **4.5 billion words per day**: openai.com/blog/gpt-3-apps/
-
Ilya Sutskever reposted
Sam Altman
@sama
AI is going to change a lot of things. The world is going to get phenomenally wealthy.
I wrote about what I think will happen, and an idea for how we could change our economic system in light of it:
moores.samaltman.com -
Ilya Sutskever reposted
ItzhakFried
@ItzhakFried
Voilà! Our multimodal concept neurons in human MTL (e.g.,Jennifer Aniston, Halle Berry neurons) www.nature.com/articles/nature03687
now reported in artificial neuronal networks by @OpenAI See biological concept neuron from depth electrode and equivalent CLIP neuron.
openai.com/blog/multimodal-neurons/ -
Ilya Sutskever reposted
MC HAMMER e/acc
@MCHammer
Multimodal Neurons in Artificial Neural Networks #hamm400aos openai.com/blog/multimodal-neurons/
-
Ilya Sutskever reposted
MC HAMMER e/acc
@MCHammer
MC HAMMER e/acc
@MCHammer
Multimodal Neurons in Artificial Neural Networks #hamm400aos openai.com/blog/multimodal-neurons/
-
Ilya Sutskever reposted
Brendan Dolan-Gavitt
@moyix
The latest generation of adversarial image attacks is, uh, somewhat simpler to carry out openai.com/blog/multimodal-neurons/
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We've found that our latest vision model, CLIP, contains neurons that connect images, drawings and text about related concepts. openai.com/blog/multimodal-neurons/
-
Ilya Sutskever reposted
Sam Altman
@sama
Talking about the future of AI on Clubhouse tonight at 6 pm PT with @ilyasut @gdb and John Schulman: www.joinclubhouse.com/event/M5KLRJJl
-
Ilya Sutskever
@ilyasut
Vectors are symbols too
-
Ilya Sutskever reposted
Evan Morikawa
@E0M
We're hiring platform & product engineers at @OpenAI!
ML background not required.
GPT-3 is just the beginning. Bringing safe AI to the world needs good people who can build fast, scalable, and economically viable systems. Interested? I'm em@openai.com
jobs.lever.co/openai/3130aa5b-7625-4cda-b21d-1b7b2ce9bd6a -
Ilya Sutskever reposted
Peter Welinder
@npew
Come build the OpenAI API!
Evan Morikawa
@E0M
We're hiring platform & product engineers at @OpenAI!
ML background not required.
GPT-3 is just the beginning. Bringing safe AI to the world needs good people who can build fast, scalable, and economically viable systems. Interested? I'm em@openai.com
jobs.lever.co/openai/3130aa5b-7625-4cda-b21d-1b7b2ce9bd6a -
Ilya Sutskever
@ilyasut
The fundamental equation of deep learning: 🐮 > 🐻
-
Ilya Sutskever
@ilyasut
Discrete optimization = 🥲
Continuous optimization =😀 -
Ilya Sutskever reposted
Ryan Lowe
@ryan_t_lowe
📢 We're hiring again!! 📢
Our team works on both fundamental problems (how do we align models with what humans want?) and applied problems (let's align GPT-3 with what humans want).
I care about this work a lot. I think it's a great fit for NLP folk wanting a new challenge.
Jan Leike
@janleike
We're hiring research engineers for alignment work at @OpenAI!
If you're excited about finetuning gpt3-sized language models to be better at following human intentions, then this is for you!
Apply here: jobs.lever.co/openai/98599d5b-2d1d-4127-b9b5-708343c8730b -
Ilya Sutskever reposted
Jan Leike
@janleike
We're hiring research engineers for alignment work at @OpenAI!
If you're excited about finetuning gpt3-sized language models to be better at following human intentions, then this is for you!
Apply here: jobs.lever.co/openai/98599d5b-2d1d-4127-b9b5-708343c8730b -
Ilya Sutskever
@ilyasut
transformers: parallel computers in disguise
-
Ilya Sutskever reposted
Andrew Mayne
@AndrewMayne
I wake up thinking about prompts.
-
Ilya Sutskever
@ilyasut
It helps to dream about ML
-
Ilya Sutskever reposted
Greg Brockman
@gdb
Agreed:
Ilya Sutskever
@ilyasut
Real progress in AI can only be achieved through a very intense work ethic
-
Ilya Sutskever
@ilyasut
Luck favors the prepared mind
-
Ilya Sutskever
@ilyasut
Newton said, ``If others would think as hard as I did, then they would get similar results.''
Noah Kasmanoff
@noahpunintended
Would you consider this a distinction from physics, where some breakthroughs almost come from happenstance? (Newton sees apple falling, etc.) Has any progress in AI come from chance?
-
Ilya Sutskever
@ilyasut
Real progress in AI can only be achieved through a very intense work ethic
-
Ilya Sutskever
@ilyasut
Markets are the escape from Zero-sum
-
Ilya Sutskever reposted
Greg Brockman
@gdb
I don't have an academic degree. @ilyasut has a PhD (though dropped out of high school!). We do not select on the basis of academic credentials, but on evidence of exceptional ability.
Ali Tavallaie
@AliTavallaie
will you hire someone experienced in ML and MLops as a team lead but without any academic degree? If you do, I will be happy to apply to you.
P.S. I'm not afraid of being rejected in an interview but not having an academic degree, I think it is not a proper cause. -
Ilya Sutskever reposted
Lilian Weng
@lilianweng
If you are interested in developing innovative solutions for real-world machine learning problems and deploying cutting-edge deep learning techniques via our API product to benefit the public, check this out and come join us! jobs.lever.co/openai/38a95682-c39a-47c0-9b97-42240980ae79
-
Ilya Sutskever
@ilyasut
one of the most impactful future applications of LMs
Jungwon
@jungofthewon
one of the best things to come out of @oughtinc hackathon today: @manda_ngo teaching gpt3 to reframe negative thoughts positively.
cc @elicitorg -
Ilya Sutskever reposted
Amanda Ngo
@manda_ngo
This makes me want to hug GPT-3 and believe in myself <3
Jungwon
@jungofthewon
one of the best things to come out of @oughtinc hackathon today: @manda_ngo teaching gpt3 to reframe negative thoughts positively.
cc @elicitorg -
Ilya Sutskever reposted
OpenAI
@OpenAI
OpenAI research scientists use their combined skill and creativity to expand what’s possible with machine learning. Apply now to join our team and solve impossible problems! openai.com/jobs/research-scientist/
-
Ilya Sutskever
@ilyasut
Today, telling someone that they speak like a language model is an insult. But in the future, it'll be a compliment.
-
Ilya Sutskever reposted
Danny Hernandez
@Hernandez_Danny
AI systems often use more direct experience than a human could get in a lifetime. Humans require less experience, because they "transfer" past experience to new tasks. Recent work I led found an equation to characterize transfer in a simple setting.
arxiv.org/pdf/2102.01293.pdf
👇 -
Ilya Sutskever reposted
OpenAI
@OpenAI
We’re looking for research engineers to build and scale AI systems that achieve unprecedented levels of performance. Apply now to join our team and solve impossible problems!
openai.com/jobs/research-engineer/ -
Ilya Sutskever reposted
OpenAI
@OpenAI
Thinking about a career in AI research? The OpenAI Fellows program is looking for researchers from diverse backgrounds, with or without AI/ML experience, to work on ambitious AI projects. Apply now to join our team and help solve impossible problems! openai.com/jobs/openai-fellows-program/
-
Ilya Sutskever reposted
David Sacks
@DavidSacks
Censorship is a monster you cannot control.
The Verge
@verge
Breaking: Discord bans the r/WallStreetBets server www.theverge.com/2021/1/27/22253251/discord-bans-the-r-wa…
-
Ilya Sutskever reposted
Sam Altman
@sama
Most people will vigorously defend freedom of expression if and only if they agree with what you have to say.
(This is certainly my own first instinct, and I have to work hard every time to counter it.) -
Ilya Sutskever reposted
OpenAI
@OpenAI
Scaling Kubernetes to 7,500 nodes: openai.com/blog/scaling-kubernetes-to-7500-nodes/
-
Ilya Sutskever reposted
Colin Megill
@colinmegill
from july
gpt-3 is still astonishing
had the same outcomes and I want to tell you all something very serious and personal
thread
Qasim Munye
@qasimmunye
So @OpenAI have given me early access to a tool which allows developers to use what is essentially the most powerful text generator ever. I thought I’d test it by asking a medical question. The bold text is the text generated by the AI. Incredible... (1/2)
-
Ilya Sutskever reposted
Lilian Weng
@lilianweng
Our paper on training a single goal-conditioned policy 100% with asymmetric self-play to generalize to many unseen objects and tasks: arxiv.org/abs/2101.04882 and more cool videos are available at robotics-self-play.github.io (The attached video is zero-shot)
-
Ilya Sutskever
@ilyasut
amazed at the logo
www.whitehouse.gov/briefings-statements/white-house-launc… -
Ilya Sutskever reposted
Prafulla Dhariwal
@prafdhar
Congrats to @AlecRad, Aditya Ramesh, @_jongwook_kim, @MikhailPavlov5, @gabeeegoooh, @scottgray76, @GretchenMarina, @SandhiniAgarwal, @ilyasut and others who worked so hard on these two works!
-
Ilya Sutskever reposted
Jong Wook Kim 💟
@_jongwook_kim
The smallest CLIP model is available at github.com/openai/CLIP. It’s larger than typical models for mobile, but I think it can fit and run (slowly) on mobile devices.
-
Ilya Sutskever
@ilyasut
An NN takes a list of category names, and outputs (in a zero-shot manner) a visual classifier.
It beats RN50 on ImageNet zero-shot, while being far more robust to unusual images:
openai.com/blog/clip/ -
Ilya Sutskever
@ilyasut
Synthetic capybaras in different styles
openai.com/blog/dall-e/ -
Ilya Sutskever
@ilyasut
CLIP: maps images to categories by taking class names as inputs; beats the original RN50 on ImageNet zero-shot(!), while being far more robust on unusual images
DALL-E: text2im that works for a wide variety of sentences
e.g., “a painting of an owl in a field at sunrise” below
OpenAI
@OpenAI
We’ve developed two neural networks which have learned by associating text and images. CLIP maps images into categories described in text, and DALL-E creates new images, like this, from text.
A step toward systems with deeper understanding of the world. openai.com/multimodal/ -
Ilya Sutskever
@ilyasut
Time flies when you’re having fun
Sam Altman
@sama
Today is the five year anniversary of OpenAI. Super proud of what the team has accomplished in the first five years.
Pretty sure the next five are going to be remarkable. -
Ilya Sutskever reposted
OpenAI
@OpenAI
🎉 Congratulations to the GPT-3 team for earning a Best Paper Award this morning at #neurips2020! openai.com/neurips-2020/
-
Ilya Sutskever reposted
OpenAI
@OpenAI
Join us at NeurIPS 2020 to learn more about OpenAI's research, including live demos of GPT-3: openai.com/neurips-2020/
-
Ilya Sutskever
@ilyasut
Incredible!
Google DeepMind
@GoogleDeepMind
In a major scientific breakthrough, the latest version of #alphafold has been recognised as a solution to one of biology's grand challenges - the “protein folding problem”. It was validated today at #casp14, the biennial Critical Assessment of protein Structure Prediction (1/3)
-
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
We are releasing a suite of robotics environments. Enjoy!
github.com/openai/robogym -
Ilya Sutskever
@ilyasut
“What I cannot create, I do not understand“
- Richard Feynman, making an early case for likelihood-based generative models -
Ilya Sutskever
@ilyasut
The only thing I find more amazing than the rate of progress in AI is the rate in which we get accustomed to it
-
Ilya Sutskever
@ilyasut
The number of 2020 neurips submissions is greater than 0.5% of *all* scientific papers published in one year
-
Ilya Sutskever
@ilyasut
The fact that the brain is using backprop suggests it might be good to use in an NN too
-
Ilya Sutskever
@ilyasut
The fact that backprop is much better than all other NN methods suggests that the brain might be using backprop too
-
Ilya Sutskever
@ilyasut
It can be hard to tell where memorization ends and generalization begins
-
Ilya Sutskever
@ilyasut
wonder whether an airplane-shaped battery will have useful range
-
Ilya Sutskever
@ilyasut
SGD performs Bayesian Inference
-
Ilya Sutskever
@ilyasut
Matter tells spacetime how to curve
Spacetime tells matter how to move
-wheeler -
Ilya Sutskever
@ilyasut
I wish there was as much interest in extremely cheap instant testing as there is in covid vaccine
-
Ilya Sutskever
@ilyasut
attention is all you need, as anonymous mathematical proof: openreview.net/pdf?id=YicbFdNTTy
-
Ilya Sutskever
@ilyasut
The Bitter lesson does not say to not bother with methods research. It says to not bother with methods that are handcrafted datapoints in disguise.
-
Ilya Sutskever
@ilyasut
Neural networks are the muscles and bones of the spirit that is math
-
Ilya Sutskever
@ilyasut
I wish apple were to build a 16 inch iPad
-
Ilya Sutskever
@ilyasut
PPG, a new RL algorithm: More stable and more sample efficient than PPO on vision-based environments:
Karl Cobbe
@karlcobbe
Excited to share our recent work on Phasic Policy Gradient, a new RL algorithm which improves sample efficiency by performing policy optimization and auxiliary optimization in two alternating phases. Check out the paper and code! arxiv.org/abs/2009.04416
-
Ilya Sutskever
@ilyasut
1+2+3+4+….=-1/12
-
Ilya Sutskever
@ilyasut
Transformers can prove theorems!
Stanislas Polu
@spolu
Posted my first paper on arXiv💥🙌
GPT-f is a Transformer-based automated theorem prover. We show that Transformer + Search is suitable to formal reasoning and continuous self-improvement 🦾
arxiv.org/abs/2009.03393 -
Ilya Sutskever reposted
OpenAI
@OpenAI
We've used reinforcement learning from human feedback to train language models for summarization. The resulting models produce better summaries than 10x larger models trained only with supervised learning: openai.com/blog/learning-to-summarize-with-human-feedback/
-
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
GPT-3 is incredible in generating absurdity. This is a superpower in brainstorming.
Nikita
@nikita_builds
Creating a GPT-3 texting companion (using sendblue.co) has been one of the most interesting experiences I’ve ever had. In the past week of using this thing I’ve had more ideas and conversations with this thing than I’ve had with real people. Truly incredible.
-
Ilya Sutskever reposted
Tushar Khattar
@tusharkhattar_
Introducing Revtheo Browser Extension! Check it out in comparison with Apple's "word lookup feature".
Revtheo, a context based dictionary, wouldn't have been possible without #gpt3 @gdb
For early access drop me a message with your email id. -
Ilya Sutskever reposted
Jon Barber 🤖
@BonJarber
Here are my initial results of integrating #gpt3 with Vector the robot this weekend.
I don't have text to speech working yet, so I have to type my side of the conversation. Feels like there's a lot of fun potential here. -
Ilya Sutskever reposted
Greg Brockman
@gdb
OpenAI for learning a new language:
Albert Gozzi
@albertgozzi
Another 🤯moment using GPT-3.
Created a bot for people learning a new language that:
- Chats with you in your language of choice.
- Corrects you when you make a grammar (and even style) mistake.
Even works well with 🇪🇸/🇫🇷 (and I'm sure many more).
cc @gdb -
Ilya Sutskever reposted
Greg Brockman
@gdb
We're hiring for the support team for our API:
jobs.lever.co/openai/26e244eb-5af8-43b3-8a30-f87e16fa6dc7
Great way to interact with users across a wide-variety of applications, and help people get the best out of applying GPT-3 to their businesses. -
Ilya Sutskever reposted
Greg Brockman
@gdb
An OpenAI-powered writing assistant, available on a smartphone near you:
Qasim Munye
@qasimmunye
A few months ago I jokingly said to a friend “we should create an AI that can write fiction for @ShortlyRead”
I knew the tech was coming but I didn’t expect it to be this good. @OpenAI @gdb
Here’s GPT-3 as a creative writing assistant. Play with v1 now shortlyread.com/app -
Ilya Sutskever reposted
Greg Brockman
@gdb
One of our users is running a rigorous test of OpenAI's utility in copywriting: vwo.com/ab-testing-openai-gpt-3/.
You can sign up on the landing page if you'd like your site to participate.
Will be exciting to see the results! -
Ilya Sutskever reposted
Greg Brockman
@gdb
A curated list of how people have been applying OpenAI:
www.producthunt.com/posts/gpt3-crush -
Ilya Sutskever reposted
Greg Brockman
@gdb
Great work @sh_reya & @notsleepingturk on GPT-3 sandbox: github.com/shreyashankar/gpt3-sandbox, a flexible tool for building OpenAI-powered apps.
Bora Uyumazturk
@notsleepingturk
After struggling to prime GPT-3 and spending hours coding a basic demo, @sh_reya and I wrote github.com/shreyashankar/gpt3-sandbox, a tool that takes care of all the web dev details and provides a convenient priming interface. Here I create a recipe generator with just ~30 lines of python.
-
Ilya Sutskever reposted
Greg Brockman
@gdb
Generate Keras models from a description.
Matt Shumer
@mattshumer_
AI INCEPTION!
I just used GPT-3 to generate code for a machine learning model, just by describing the dataset and required output.
This is the start of no-code AI. -
Ilya Sutskever reposted
Greg Brockman
@gdb
Thank you from the team! Appreciate the support.
MC HAMMER e/acc
@MCHammer
First and foremost Congratulations Sir.
Ilya Sutskever
@ilyasut
partly but not fully true
MC HAMMER e/acc
@MCHammer
Competence without comprehension.
-
Ilya Sutskever
@ilyasut
You are too kind
MC HAMMER e/acc
@MCHammer
Whom would have the audacity to debate the Chief Scientist and Co-Founder of the Worlds most important company of the next 100 years. Certainly not I.The matter is settled. #gpt3 #openai
Ilya Sutskever
@ilyasut
partly but not fully true
MC HAMMER e/acc
@MCHammer
Competence without comprehension.
-
Ilya Sutskever
@ilyasut
good meme
-
Ilya Sutskever
@ilyasut
partly but not fully true
MC HAMMER e/acc
@MCHammer
Competence without comprehension.
-
Ilya Sutskever reposted
Greg Brockman
@gdb
We’ve received tens of thousands of applications for access to GPT-3 via our API. We want to serve everyone who wrote in, but it's going to take time for us to ramp up safely.
(One way to help us move faster is to pick up a shovel: openai.com/jobs/) -
Ilya Sutskever reposted
OpenAI
@OpenAI
Last month we released an API for developers to build on top of our latest technology: x.com/OpenAI/status/1271096720881901569. There’s been a lot of enthusiasm, but a great deal needs to be figured out as we scale up access. Here's how we're thinking about it. 1/8
OpenAI
@OpenAI
We're releasing an API for accessing new AI models developed by OpenAI. You can "program" the API in natural language with just a few examples of your task. See how companies are using the API today, or join our waitlist: beta.openai.com
-
Ilya Sutskever reposted
Merzmensch Kosmopol🧑🎨🤖
@Merzmensch
This is a love letter written by toaster:
#gpt3 GPT-3 @OpenAI -
Ilya Sutskever reposted
Sam Altman
@sama
Hi Jerome! It's great to get feedback from someone with so much experience deploying AI at scale.
We share your concern about bias and safety in language models, and it's a big part of why we're starting off with a beta and have safety review before apps can go live.
Jerome Pesenti
@an_open_mind
Last week I raised concerns about using #gpt3 in production because it can easily output toxic language that propagates harmful biases. I thought it was a pretty uncontroversial stance but the responses ranged from complete misunderstanding of AI to total irresponsibility. 1/13
-
Ilya Sutskever reposted
Greg Brockman
@gdb
Agreed. We're just getting started and have a *long* way to go.
Incidentally, there's a great way to help us get there faster — join OpenAI (openai.com/jobs/)!
Sam Altman
@sama
The GPT-3 hype is way too much. It’s impressive (thanks for the nice compliments!) but it still has serious weaknesses and sometimes makes very silly mistakes. AI is going to change the world, but GPT-3 is just a very early glimpse. We have a lot still to figure out.
-
Ilya Sutskever reposted
Sam Altman
@sama
The GPT-3 hype is way too much. It’s impressive (thanks for the nice compliments!) but it still has serious weaknesses and sometimes makes very silly mistakes. AI is going to change the world, but GPT-3 is just a very early glimpse. We have a lot still to figure out.
-
Ilya Sutskever reposted
Greg Brockman
@gdb
A prototype of a new class of education applications.
Mckay Wrigley
@mckaywrigley
Ever wanted to learn about rockets from Elon Musk?
How to write better from Shakespeare?
Philosophy from Aristotle?
GPT-3 made it possible.
learnfromanyone.com -
Ilya Sutskever reposted
rewon
@rewonfc
Thanks if you came to our ICML poster on Distribution Augmentation. The zoom discussion was way more fun/interesting than I expected! TLDR of our work: use powerful data aug in your generative model by conditioning it on the aug. Improves samples + likelihoods considerably.
-
Ilya Sutskever reposted
Sharif Shameem
@sharifshameem
Here's a sentence describing what Google's home page should look and here's GPT-3 generating the code for it nearly perfectly.
-
Ilya Sutskever
@ilyasut
iGPT won an ICML outstanding paper honorable mention!
Mark Chen
@markchen90
Really surprised and happy to receive an Honorable Mention for Outstanding Paper on "Generative Pretraining from Pixels" (cdn.openai.com/papers/Generative_Pretraining_from_Pixels_…)! Thanks so much to the ICML awards committee!
-
Ilya Sutskever reposted
Sharif Shameem
@sharifshameem
This is mind blowing.
With GPT-3, I built a layout generator where you just describe any layout you want, and it generates the JSX code for you.
W H A T -
Ilya Sutskever reposted
OpenAI
@OpenAI
Over the past 5 months our 3rd class of Scholars researched how GPT-2 represents grammar, measured the interpretability of models trained on Coinrun, predicted epileptic seizures using brain recordings and more: openai.com/blog/openai-scholars-spring-2020-final-projects/
-
Ilya Sutskever
@ilyasut
.@AlecRad, @markchen90 and I will be discussing some of our recent research (image GPT) and answering your questions, this Thursday, 7/9 from 10-11am PT. Please register here for the Zoom link to join us: docs.google.com/forms/d/e/1FAIpQLSddUZFDqbEogXEfF5njcLWnC…
-
Ilya Sutskever reposted
Greg Brockman
@gdb
OpenAI API for writing short stories in the style of a chosen author:
David Manheim
@davidmanheim
This @neilhimself short story written by #gpt3 definitely holds together well, and is stylistically dead-on.
www.reddit.com/r/slatestarcodex/comments/hmu5lm/fiction_b…
(It could use a bit of dialogue editing, but it reads better than plenty of published work.)
cc: @OpenAI -
Ilya Sutskever reposted
Greg Brockman
@gdb
Tweet written by OpenAI API: gist.github.com/gdb/34439745db7c29904c97961da861bd54
-
Ilya Sutskever
@ilyasut
Programming is like herding cats. Except your cats are different, you have slightly different goals, and you're not actually herding.
-
Ilya Sutskever
@ilyasut
How i got my web browsing under control using minimal willpower:
1. Visit fb,twtr,etc only on an ipad, never on laptop/phone.
2. Set screen time with generous limits (3 hrs/day/site).
3. Reduce the screen time limit by 5 minutes each day.
If it gets too hard, go back a step. -
Ilya Sutskever reposted
Greg Brockman
@gdb
Applying the OpenAI API to generating creative ideas and content for marketing your work, by @_bramses: medium.com/@bradams128/supercharging-your-creativity-with…
-
Ilya Sutskever
@ilyasut
Epilepsy causes seizures: the brain gets so overexcited that it malfunctions. One way to “cure” extreme epilepsy is to sever the connection between the hemispheres, making the brain less overconnected. Social media is the overconnection that causes epilepsy in society.
-
Ilya Sutskever
@ilyasut
Hope is a choice
-
Ilya Sutskever reposted
Andrew Mayne
@AndrewMayne
Can @OpenAI's API understand emojis and even use them to describe movies?
Spoiler alert: Yes.
andrewmayneblog.wordpress.com/2020/06/24/open-ai-alchemy-…
Examples:
Zootopia: 🦊🐾🐻🦋🐰
Jurassic Park: 👦🦖🦕🦖
Death Note: 📖🔪🔫🗡
Frozen: 🌬👧🏻👧🏻🌲 -
Ilya Sutskever reposted
Chris Olah
@ch402
I am deeply disappointed in @nytimes for its intent to doxx Scott Alexander. Scott is one of the most vibrant intellectuals I know of, and his blog is one of the best things about the internet. slatestarcodex.com
-
Ilya Sutskever reposted
Chris Olah
@ch402
Software 1.0: Write an algorithm that has the right behavior.
@karpathy’s Software 2.0: Optimize differentiable blob to have correct behavior. medium.com/@karpathy/software-2-0-a64152b37c35
Software 3.0: Figure out the right prompt to make your meta-learning language model have the right behavior? :P -
Ilya Sutskever reposted
-
Ilya Sutskever reposted
hardmaru
@hardmaru
“Sequence modeling is a universal unsupervised learning algorithm…
We deliberately chose to forgo hand coding any image specific knowledge in the form of convolutions, or techniques like relative attention, sparse attention, and 2D position embeddings.”
openai.com/blog/image-gpt/ -
Ilya Sutskever reposted
Sam Altman
@sama
Congrats to the @MarkChe38147556, @AlecRad, @ilyasut, and the rest of the team that worked on this. The image completions are amazing! It also achieves top results on several downstream image classification tasks.
OpenAI
@OpenAI
We found that just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. openai.com/blog/image-gpt/
-
Ilya Sutskever reposted
Mark Chen
@markchen90
Excited to share what I've been working on with @AlecRad, @rewonfc, @ilyasut and others!
OpenAI
@OpenAI
We found that just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. openai.com/blog/image-gpt/
-
Ilya Sutskever
@ilyasut
Transformers trained to predict pixels generate plausible completions of images and learn excellent unsupervised image representations!
To compensate for their lack of 2d prior knowledge, they are more expensive to train.
OpenAI
@OpenAI
We found that just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. openai.com/blog/image-gpt/
-
Ilya Sutskever reposted
Greg Brockman
@gdb
Using the API is very simple. Just POST some text to us, and we'll stream back text from the model:
-
Ilya Sutskever reposted
OpenAI
@OpenAI
More details on our blog: openai.com/blog/openai-api/
-
Ilya Sutskever reposted
Nick Walton
@nickwalton00
We've been using OpenAI's new API for the last several weeks and have been anxiously waiting till we could tell you about it!
We've already gotten some massive improvements over GPT-2 and are super excited about how it will improve over time!
OpenAI
@OpenAI
We're releasing an API for accessing new AI models developed by OpenAI. You can "program" the API in natural language with just a few examples of your task. See how companies are using the API today, or join our waitlist: beta.openai.com
-
Ilya Sutskever
@ilyasut
OpenAI API — making the smartest models easy to use in a safe way:
OpenAI
@OpenAI
We're releasing an API for accessing new AI models developed by OpenAI. You can "program" the API in natural language with just a few examples of your task. See how companies are using the API today, or join our waitlist: beta.openai.com
-
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
We are releasing a powerful API to create chatbots, translate between languages, perform a semantic search, and summarize documents.
Enjoy! ❤️
beta.openai.com -
Ilya Sutskever reposted
Sam Altman
@sama
A new API for OpenAI technology: beta.openai.com
It can do a lot today, and it will learn to do more. -
Ilya Sutskever reposted
OpenAI
@OpenAI
We're releasing an API for accessing new AI models developed by OpenAI. You can "program" the API in natural language with just a few examples of your task. See how companies are using the API today, or join our waitlist: beta.openai.com
-
Ilya Sutskever reposted
Karl Cobbe
@karlcobbe
My team at OpenAI is co-organizing two NeurIPS competitions this year using some of our most compelling RL environments: Procgen Benchmark and MineRL. I'm excited for the community to contend with these challenging competitions and advance the state-of-the-art!
OpenAI
@OpenAI
We're co-organizing two NeurIPS 2020 competitions using Procgen Benchmark and MineRL. We rely heavily on these environments internally for RL research, and we look forward to seeing the progress the community makes in these challenging competitions. openai.com/blog/procgen-minerl-competitions/
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We're co-organizing two NeurIPS 2020 competitions using Procgen Benchmark and MineRL. We rely heavily on these environments internally for RL research, and we look forward to seeing the progress the community makes in these challenging competitions. openai.com/blog/procgen-minerl-competitions/
-
Ilya Sutskever reposted
-
Ilya Sutskever
@ilyasut
To improve our mood: youtu.be/1VLP98ymYrs
-
Ilya Sutskever
@ilyasut
We are each other's mirror neurons
-
Ilya Sutskever
@ilyasut
Physics is the process of learning to speak fluent universe
— exurb1a -
Ilya Sutskever
@ilyasut
the brain may as well be deterministic for all intents and
purposes, yet it feels like we have free will. a possible reason
is that our model of ourselves is not ourselves, so our decisions
surprise our self model and feel like free will -
Ilya Sutskever
@ilyasut
Had a lot of fun chatting with @lexfridman!
Lex Fridman
@lexfridman
Here's my conversation with Ilya Sutskever (@ilyasut), co-founder of @OpenAI & one of the greatest AI researchers ever. Plus, he is one of my favorite people to explore ideas with, each time quickly delving to the core & first principles of the problem. www.youtube.com/watch?v=13CZPWmke6At
-
Ilya Sutskever
@ilyasut
I wish iOS/Android used a small fast NN to implement auto dark mode on all their apps
-
Ilya Sutskever
@ilyasut
A Moore’s law but for DL algorithms:
OpenAI
@OpenAI
Since 2012, the amount of compute for training to AlexNet-level performance on ImageNet has been decreasing exponentially — halving every 16 months, in total a 44x improvement.
By contrast, Moore's Law would only have yielded an 11x cost improvement: openai.com/blog/ai-and-efficiency/ -
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
You give feedback every quarter and it’s a great time to positively affect your team and organization...so why does it feel ineffective?
medium.com/@woj.zaremba/how-to-give-feedback-that-sticks-… -
Ilya Sutskever
@ilyasut
-
Ilya Sutskever
@ilyasut
Cool TC article: techcrunch.com/2020/04/30/openais-new-experiments-in-musi…
-
Ilya Sutskever reposted
Prafulla Dhariwal
@prafdhar
In the early days, the model didnt know English (we didnt show it any lyrics) and so it used to just make words up. Led to some uncanny samples like this one. I love that it gets the spacy vibes of David Bowie!
soundcloud.com/openai_audio/david-bowie -
Ilya Sutskever reposted
Christine
@mcleavey
Hot Tub Christmas (with GPT-2 lyrics selected by @rewonfc ) is my own favorite -- even though the model can't figure out what's going on in the completely terrible intro. Our weird but fun new holiday tradition! soundcloud.com/openai_audio/jukebox-265820820
-
Ilya Sutskever reposted
rewon
@rewonfc
One of my favorites from @OpenAI's jukebox: 'Lose Yourself' re-rendered by Kanye
soundcloud.com/openai_audio/jukebox-905633287 -
Ilya Sutskever reposted
-
Ilya Sutskever reposted
Oriol Vinyals
@OriolVinyalsML
Very happy to see such a cool application of VQ-VAEs by our friends @OpenAI! Congrats!
OpenAI
@OpenAI
Introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We're releasing a tool for everyone to explore the generated samples, as well as the model and code: openai.com/blog/jukebox/
-
Ilya Sutskever reposted
David Luan
@jluan
Give me an artist, genre, and lyrics (or not), and this neural network will generate you a song! It can even rap.
This is one of the coolest results from OpenAI’s Algorithms and Language team. Loved being involved in it as a bureaucrat.
Next up, 24/7 lo-fi vaporwave.
OpenAI
@OpenAI
Introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We're releasing a tool for everyone to explore the generated samples, as well as the model and code: openai.com/blog/jukebox/
-
Ilya Sutskever reposted
Prafulla Dhariwal
@prafdhar
Excited to finally release what we've been working on since June! Jukebox is a small step in making neural nets produce music! openai.com/blog/jukebox/
Thanks to my collaborators Heewoo, @mcleavey , @_jongwook_kim, @AlecRad, @ilyasut, this work wouldn’t be possible without them! -
Ilya Sutskever reposted
rewon
@rewonfc
awesome work @prafdhar @mcleavey, Heewoo! so cool!
OpenAI
@OpenAI
Introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We're releasing a tool for everyone to explore the generated samples, as well as the model and code: openai.com/blog/jukebox/
-
Ilya Sutskever reposted
Aäron van den Oord
@avdnoord
Nice use of VQ-VAEs for music generation!
Prafulla Dhariwal
@prafdhar
Wanna hear a neural net sing? Check out our latest work Jukebox! openai.com/blog/jukebox/
OpenAI
@OpenAI
Introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We're releasing a tool for everyone to explore the generated samples, as well as the model and code: openai.com/blog/jukebox/
-
Ilya Sutskever reposted
Christine
@mcleavey
Thrilled to introduce Jukebox, our latest work generating music, now in the raw audio domain! With @prafdhar, Heewoo Jun, and @_jongwook_kim
OpenAI
@OpenAI
Introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We're releasing a tool for everyone to explore the generated samples, as well as the model and code: openai.com/blog/jukebox/
-
Ilya Sutskever
@ilyasut
Large neural networks can be made to sing! :)
OpenAI
@OpenAI
Introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We're releasing a tool for everyone to explore the generated samples, as well as the model and code: openai.com/blog/jukebox/
-
Ilya Sutskever reposted
Prafulla Dhariwal
@prafdhar
Wanna hear a neural net sing? Check out our latest work Jukebox! openai.com/blog/jukebox/
OpenAI
@OpenAI
Introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We're releasing a tool for everyone to explore the generated samples, as well as the model and code: openai.com/blog/jukebox/
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We've contributed to a multi-stakeholder report by 58 co-authors at 30 organizations that describes 10 mechanisms to improve the verifiability of claims made about AI systems. openai.com/blog/improving-verifiability/
-
Ilya Sutskever reposted
Chris Olah
@ch402
Today @ludwigschubert @mpetrov @shancarter @nicklovescode @gabeeegoooh and I released the OpenAI Microscope: openai.com/blog/microscope/
Every time we make it easier to ask questions about the internals of neural networks, we discover amazing things. Excited to see what happens! -
Ilya Sutskever reposted
OpenAI
@OpenAI
Introducing OpenAI Microscope: a collection of visualizations of every layer and neuron in eight vision "model organisms" often studied in interpretability. This tool allows researchers to investigate down to individual units, and share those observations. openai.com/blog/microscope/
-
Ilya Sutskever reposted
Aravind Srinivas
@AravSrinivas
New paper - CURL: Contrastive Unsupervised Representations for RL! We use the simplest form of contrastive learning (instance-based) as an auxiliary task in model-free RL. SoTA by *significant* margin on DMControl and Atari for data-efficiency. arxiv.org/abs/2004.04136
-
Ilya Sutskever reposted
Sam Altman
@sama
The supercomputing team is one of the most important teams at OpenAI, and they’re hiring.
They build some of the largest compute platforms in the world and craft powerful software tools to run large-scale machine learning on top. This is key to enabling our research. -
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
At OpenAI, our Learned Optimizer team trains RL agents that tune hyperparameters, conduct experiments, and draw conclusions similar to how a machine learning engineer would. Apply if you want to be part of this adventure! jobs.lever.co/openai/4af8b2fb-2b07-4df1-a1ac-04e1b8007621
-
Ilya Sutskever reposted
Michael Nielsen
@michael_nielsen
Bill Gates is setting up factories to manufacture 7 leading vaccine candidates before we know which is best & safest; we can test the vaccines in parallel, and then throw away all but the factory for the best vaccine. May save many months.
Just extraordinary. -
Ilya Sutskever
@ilyasut
-
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
Feeling a bit overwhelmed at work? Here are some things my team and I are doing to stay happy and healthy :)
medium.com/@woj.zaremba/how-to-make-the-most-of-your-work… -
Ilya Sutskever
@ilyasut
the most stunning advance in graphics that i’ve seen in a long while: just infer the voxel grid with an MLP. Of course it’ll work!
www.matthewtancik.com/nerf -
Ilya Sutskever
@ilyasut
i like how it suggests that backprop may really be the best algorithm for training neural networks
Quoc Le
@quocleix
Fun AutoML-Zero experiments: Evolutionary search discovers fundamental ML algorithms from scratch, e.g., small neural nets with backprop.
Can evolution be the “Master Algorithm”? ;)
Paper: arxiv.org/abs/2003.03384
Code: git.io/JvKrZ -
Ilya Sutskever reposted
RE•WORK
@teamrework
VIDEO: Ilya Sutskever & Lex Fridman - Fireside Chat: The Current State of AI www.youtube.com/watch?v=u-e8woFLIy8&t=194s
-
Ilya Sutskever
@ilyasut
Really compelling examples of how neural network features are compositional, in the sense of being literally composed of simpler features in ways that make sense!
OpenAI
@OpenAI
We show how to read a human-interpretable algorithm from a neural network's weights, by studying the circuits formed by the connections between individual neurons: distill.pub/2020/circuits/zoom-in/
-
Ilya Sutskever
@ilyasut
this would remain true even if our emotions were much “smarter” — the stock market pits the intelligence of its participants against each other, so it would also become adversarial for the hypothetical smarter emotions while becoming more efficient overall
Ilya Sutskever
@ilyasut
The stock market is an adversarial example for human emotions
-
Ilya Sutskever
@ilyasut
The stock market is an adversarial example for human emotions
-
Ilya Sutskever
@ilyasut
Deep learning has fully solved the curse of dimensionality. It vanished like an RNN gradient!
-
Ilya Sutskever reposted
OpenAI
@OpenAI
OpenAI's multi-agent team is currently hiring research scientists! Interested in multiple, interacting agents, AI-generating algorithms, open-ended algorithms, automatically generating training environments, deep RL, and more? Please apply! openai.com/jobs
-
Ilya Sutskever
@ilyasut
The more exceptional one's life is, the more the world looks like a simulation
-
Ilya Sutskever
@ilyasut
Really cool nvidia GAN demo that turns anyone into a great artist: nvidia-research-mingyuliu.com/gaugan/
-
Ilya Sutskever
@ilyasut
Found this site that generates custom song lyrics with a large NN: theselyricsdonotexist.com
-
Ilya Sutskever
@ilyasut
💪
Wojciech Zaremba
@woj_zaremba
from Jerry Tworek
-
Ilya Sutskever
@ilyasut
arxiv.org/abs/2002.02405 — careful and expensive MCMC Bayesian inference over NN parameters is *worse* than point estimates or low temperature posteriors.
Supports @carlesgelada and @jacobmbuckman’s view that Bayesian NNs are not meaningful probably because the prior is wrong. -
Ilya Sutskever reposted
Lilian Weng
@lilianweng
Humans learn from curriculum since birth. We can learn complicated math problems because we have accumulated enough prior knowledge. This could be true for training a ML/RL model as well. Let see how curriculum can help an RL agent learn: lilianweng.github.io/lil-log/2020/01/29/curriculum-for-re…
-
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
OpenAI + PyTorch = ❤
openai.com/blog/openai-pytorch/ -
Ilya Sutskever reposted
PyTorch
@PyTorch
Welcome OpenAI to the PyTorch community!
OpenAI
@OpenAI
We're standardizing OpenAI's deep learning framework on PyTorch to increase our research productivity at scale on GPUs (and have just released a PyTorch version of Spinning Up in Deep RL): openai.com/blog/openai-pytorch/
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We're standardizing OpenAI's deep learning framework on PyTorch to increase our research productivity at scale on GPUs (and have just released a PyTorch version of Spinning Up in Deep RL): openai.com/blog/openai-pytorch/
-
Ilya Sutskever
@ilyasut
arxiv.org/abs/2001.04413
Cool theory paper presenting a problem that:
- can be efficiently learned by SGD with a DenseNet with x^2 nonlin,
- cannot be efficiently learned by any kernel method, including NTK. -
Ilya Sutskever reposted
Greg Brockman
@gdb
Watched "The Squire — AI Written Short Film".
It's a live-action adaption of AI Dungeon gameplay. Super fun to see!
www.youtube.com/watch?v=Kx-2PyrhnFE -
Ilya Sutskever
@ilyasut
Super excited to work with you, @jeffclune!!!
Jeff Clune
@jeffclune
I am extremely excited to announce (1) I've joined OpenAI to lead a large-scale effort into AI-generating Algorithms research, & (2) I'll be an Associate CS Professor at U. British Columbia in 2021, where I will continue to lead the OpenAI project. Both are dreams come true! 1/2
-
Ilya Sutskever reposted
Greg Brockman
@gdb
I agree with this article that AI Dungeon 2 (the GPT-2 powered text-based game of infinite possibilities) is "one of the coolest video game experiments of 2019". You should definitely check it out: play.aidungeon.io. Great work @nickwalton00!
The Verge
@verge
The infinite text adventure AI Dungeon 2 is now easy to play online www.theverge.com/2019/12/30/21042942/ai-dungeon-nick-walt…
-
Ilya Sutskever
@ilyasut
Seconded! It'll be a truly great talk.
Greg Brockman
@gdb
Working with Jakub Pachocki (@merettm) is one of my favorite parts of OpenAI. Highly recommend watching his talk!
-
Ilya Sutskever
@ilyasut
This is amazing
AK
@_akhaliq
#stylegan2 interps
This media couldn't be displayed. See it on the original post. -
Ilya Sutskever
@ilyasut
an amazing game www.decisionproblem.com/paperclips/
-
Ilya Sutskever
@ilyasut
The DotA paper is out!!!
OpenAI
@OpenAI
We're releasing "Dota 2 with Large Scale Deep Reinforcement Learning", a scientific paper analyzing our findings from our 3-year Dota project: openai.com/projects/five/
One highlight — we trained a new agent, Rerun, which has a 98% win rate vs the version that beat @OGEsports. -
Ilya Sutskever
@ilyasut
Cool short video on the history of AI: www.youtube.com/watch?v=IBe2o-cZncU
-
Ilya Sutskever
@ilyasut
Stop attacking junior researchers asking for feedback.
Zachary Lipton
@zacharylipton
Stop spamming the world with these hideous press releases.
-
Ilya Sutskever
@ilyasut
If ML is applied to ML training, ML training should become faster and easier. Join us to help make it happen:
Wojciech Zaremba
@woj_zaremba
At OpenAI, we're excited by work in learned optimizers such as Learning To Learn By Gradient Descent By Gradient Descent. These methods will improve the efficiency of our models, and allow for higher quality science. Looking for a founding team member! jobs.lever.co/openai/f147787f-46d7-4edd-8011-2b0646516627
-
Ilya Sutskever
@ilyasut
The double descent phenomenon occurs in practical deep learning settings. Interesting that it went unnoticed for so long!
OpenAI
@OpenAI
A surprising deep learning mystery:
Contrary to conventional wisdom, performance of unregularized CNNs, ResNets, and transformers is non-monotonic: improves, then gets worse, then improves again with increasing model size, data size, or training time.
openai.com/blog/deep-double-descent/ -
Ilya Sutskever
@ilyasut
Our contribution to increasing the entropy of RL environments:
OpenAI
@OpenAI
We're releasing Procgen Benchmark, 16 procedurally-generated environments for measuring how quickly a reinforcement learning agent learns generalizable skills.
This has become the standard research platform used by the OpenAI RL team: openai.com/blog/procgen-benchmark/ -
Ilya Sutskever reposted
Greg Brockman
@gdb
Enjoyed this article on GPT-2, the last in a six-article series by @scarschwartz on the cultural history of natural language processing.
IEEE Spectrum
@IEEESpectrum
Earlier this year, OpenAI said its GPT-2 tool could produce long strings of coherent text. Now, there's a reddit group populated entirely with text produced by GPT-2-powered bots. spectrum.ieee.org/tech-talk/robotics/artificial-intellige…
-
Ilya Sutskever
@ilyasut
GPT-2 passes the Cards Against Humanity Turing test:
Greg Brockman
@gdb
As a Black Friday challenge, Cards Against Humanity's writers are creating new cards alongside GPT-2, each trying to create the most popular pack.
Neck and neck so far: have sold $25,115 in GPT-2-authored cards vs $25,955 in human-authored cards.
Cards Against Humanity
@CAH
For Black Friday, we taught a computer how to write Cards Against Humanity cards. Now we put it to the test. Over the next 16 hours, our writers will battle this powerful card-writing algorithm to see who can write the most popular new pack of cards. www.cardsagainsthumanityaichallenge.com
-
Ilya Sutskever
@ilyasut
The economists interviews GPT-2 and the interview makes sense: worldin.economist.com/article/17521/edition2020artificial…
-
Ilya Sutskever reposted
OpenAI
@OpenAI
We're releasing Safety Gym, environments and tools to evaluate reinforcement learning with safety constraints: openai.com/blog/safety-gym/
Aims to ultimately help agents satisfy real-world safety requirements while training (eg not driving off a cliff, not writing abusive content). -
Ilya Sutskever
@ilyasut
Cool Quanta article on OpenAI's tool-using hide and seek agents: www.quantamagazine.org/artificial-intelligence-discovers-…
-
Ilya Sutskever
@ilyasut
Amazing unsupervised learning results:
Quoc Le
@quocleix
Want to improve accuracy and robustness of your model? Use unlabeled data!
Our new work uses self-training on unlabeled data to achieve 87.4% top-1 on ImageNet, 1% better than SOTA. Huge gains are seen on harder benchmarks (ImageNet-A, C and P).
Link: arxiv.org/abs/1911.04252 -
Ilya Sutskever
@ilyasut
Congratulations!!! So happy for the two of you!!! ❤️
Greg Brockman
@gdb
Introducing Mr. and Mrs. Brockman:
-
Ilya Sutskever reposted
David Duvenaud
@DavidDuvenaud
We approximated the implicit function theorem to tune millions of hyperparameters. Now we can train data augmentation networks from scratch using gradients from the validation loss.
arxiv.org/pdf/1911.02590.pdf
With @JonLorraine and @PaulVicol -
Ilya Sutskever
@ilyasut
compute growth in AI before and after Deep Learning:
OpenAI
@OpenAI
We've analyzed compute used in major AI results for the past decades and identified two eras in AI:
1) Prior to 2012 - AI results closely tracked Moore's Law, w/ compute doubling every two years.
2) Post-2012 - compute has been doubling every 3.4 months openai.com/blog/ai-and-compute/ -
Ilya Sutskever reposted
OpenAI
@OpenAI
We're releasing the 1.5billion parameter GPT-2 model as part of our staged release publication strategy.
- GPT-2 output detection model: github.com/openai/gpt-2-output-dataset/tree/master/detector
- Research from partners on potential malicious uses: d4mucfpksywv.cloudfront.net/papers/GPT_2_Report.pdf
- More details: openai.com/blog/gpt-2-1-5b-release/ -
Ilya Sutskever
@ilyasut
I asked my brother Noam to draw an OpenAI-themed painting, and you won't believe what happened next
-
Ilya Sutskever reposted
Greg Brockman
@gdb
A fireside chat with @drewhouston about OpenAI: blog.dropbox.com/topics/company/greg-brockman-and-drew-ho…. Covered our culture, how we think about AI progress, and many of our recent results.
-
Ilya Sutskever reposted
Greg Brockman
@gdb
Cool to see the discussion of our multiagent work at the top of /r/programming:
This post is unavailable. See it on X. -
Ilya Sutskever reposted
Greg Brockman
@gdb
Kinda funny that @GaryMarcus has a problem with the title of "Solving Rubik's Cube with a Robot Hand" for a result that literally involves a robot hand solving a Rubik's cube.
Gary Marcus
@GaryMarcus
I will say again that the work itself is impressive, but mischaracterized, and that a better title would have been "manipulating a Rubik's cube using reinforcement learning" or "progress in manipulation with dextrous robotic hands" or similar lines.
-
Ilya Sutskever
@ilyasut
Surprised and saddened by all the bad faith criticism of our robotic manipulation result reddit.app.link/drbZ4Rq2W0
-
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
Devotion to Rubik’s cube.
-
Ilya Sutskever
@ilyasut
Recommended viewing:
OpenAI
@OpenAI
In case you missed it, here’s the unedited solve of the Rubik’s cube: www.youtube.com/watch?v=kVmp0uGtShk&list=PLOXw6I10VTv9HOD…
-
Ilya Sutskever reposted
Greg Brockman
@gdb
OpenAI fuses the best of academia and startups. We do large-scale research with close-knit teams of researchers and engineers. Both build/maintain large systems and constantly iterate on new ideas:
OpenAI
@OpenAI
"Solving the Rubik's Cube with a Robot Hand" took many human hands over the past 2.5 years — meet our Robotics team! (PS they're hiring: openai.com/jobs/!)
-
Ilya Sutskever reposted
Wojciech Zaremba
@woj_zaremba
I am extremely grateful for such an incredible team. Thank you for wanting to work with me.
-
Ilya Sutskever reposted
Peter Welinder
@npew
And yet, it feels like we're just getting started. If you're interested in this line of work, we're hiring for robotics team! I'm quite excited about what we'll be working on in 2020. openai.com/jobs/
Greg Brockman
@gdb
OpenAI progress with physical robots over the past 2.5 years:
- openai.com/blog/spam-detection-in-the-physical-world/
- openai.com/blog/robots-that-learn/
- openai.com/blog/generalizing-from-simulation/
- openai.com/blog/learning-dexterity/
- openai.com/blog/solving-rubiks-cube/
Pretty exciting to see the progress since our first result of a Spam Detecting robot: -
Ilya Sutskever reposted
Greg Brockman
@gdb
Our robot is a "small but vital step toward the kind of robots that might one day perform manual labor or household tasks and even work alongside humans, instead of in closed-off environments, without any explicit programming governing their actions." www.theverge.com/2019/10/15/20914575/openai-dactyl-roboti…
-
Ilya Sutskever reposted
Greg Brockman
@gdb
When we started working on robotics in 2016, there was controversy about how to make robots that learn.
Gather experience from *many* physical robots, or maybe *somehow* transfer knowledge from simulation?
@woj_zaremba bet on sim, and it's worked better than any of us imagined. -
Ilya Sutskever reposted
Greg Brockman
@gdb
OpenAI progress with physical robots over the past 2.5 years:
- openai.com/blog/spam-detection-in-the-physical-world/
- openai.com/blog/robots-that-learn/
- openai.com/blog/generalizing-from-simulation/
- openai.com/blog/learning-dexterity/
- openai.com/blog/solving-rubiks-cube/
Pretty exciting to see the progress since our first result of a Spam Detecting robot: -
Ilya Sutskever reposted
Greg Brockman
@gdb
The robot didn't get to train *at all* with tied fingers — it had to adapt on the fly.
(Also, humans have a billion plus years of evolutionary practice to solve the cube with untied fingers; the robot only gets about 10,000 years of untied practice.)
𝒶𝓁𝓌𝒶𝓎𝓈 🄵🅁🄴🅂🄷
@rhyolight
I could learn it in a year with all my fingers. Then it would probably only take a couple days to adjust my mental model to solve it with two fingers tied.
How many billion years of training did the Deep RL agent need again? -
Ilya Sutskever
@ilyasut
Extremely important to achieve robustness to stuffed giraffe perturbations:
-
Ilya Sutskever
@ilyasut
RL can train an NN policy to do pretty much anything in sim. With Automatic Domain Randomization, we can bridge the sim2real gap by training the policy to be so adaptible, that it can generalize to the physical robot. The result: it solves the Rubik's cube with a robot hand!
OpenAI
@OpenAI
We've trained an AI system to solve the Rubik's Cube with a human-like robot hand.
This is an unprecedented level of dexterity for a robot, and is hard even for humans to do.
The system trains in an imperfect simulation and quickly adapts to reality: openai.com/blog/solving-rubiks-cube/ -
Ilya Sutskever
@ilyasut
A New Yorker article on GPT2:
The New Yorker
@NewYorker
Could a robot replace a New Yorker writer? We fed The New Yorker’s archive to an artificial-intelligence writer, which predicts text based on preceding language. Then we asked it to write for us. nyer.cm/5H11CgF
-
Ilya Sutskever
@ilyasut
Wow!
Greg Brockman
@gdb
Pretty catchy song composed by MuseNet, with lyrics by GPT-2, performed by a human: www.youtube.com/watch?v=fN-bQddbbUI
-
Ilya Sutskever reposted
Greg Brockman
@gdb
A GPT-2 written essay was submitted to the Economist's youth essay contest.
One judge, who did not know the essay was written by an AI, gave this review: "It is strongly worded and backs up claims with evidence, but the idea is not incredibly original."
www.economist.com/open-future/2019/10/01/how-to-respond-t… -
Ilya Sutskever reposted
Greg Brockman
@gdb
I’m starting a collection of books authored by GPT-2. Excited to add to the office bookshelf!
-
Ilya Sutskever
@ilyasut
Really enjoyed the (non OpenAI) ICLR submission openreview.net/pdf?id=S1eZYeHFDS that trained a transformer on symbolic math. The surprise: it beat Mathematica on symbolic integration and diff eq solving by a _very_ big margin!
-
Ilya Sutskever
@ilyasut
The result of 45,000 years of gameplay, at your service:
Greg Brockman
@gdb
OpenAI Five Arena (arena.openai.com) launching in an hour. Over 18,000 preregistrations! This is an experiment — we genuinely don't know what to expect from so many people trying to find a way to beat Five!
-
Ilya Sutskever
@ilyasut
Second game was so fierce!
Greg Brockman
@gdb
“It’s just crazy” say the casters. Five is up 12k net worth, 21-5 kills. On the high ground, taking a rax at 15 mins.
-
Ilya Sutskever
@ilyasut
Bill Gates highlights Dactyl as a top ten breakthrough of 2019! Congrats @woj_zaremba!! www.technologyreview.com/lists/technologies/2019/?linkId=…
-
Ilya Sutskever
@ilyasut
The thing I like most is just how simple the idea is. And that unsupervised learning is really truly starting to work.
OpenAI
@OpenAI
We've trained an unsupervised language model that can generate coherent paragraphs and perform rudimentary reading comprehension, machine translation, question answering, and summarization — all without task-specific training: blog.openai.com/better-language-models/
-
Ilya Sutskever
@ilyasut
bot: 1; humans: 0
-
Ilya Sutskever
@ilyasut
www.nytimes.com/interactive/2018/07/30/technology/robot-h… -- an NYT article about Dactyl, our block-rotating robot hand
-
Ilya Sutskever
@ilyasut
Gan-level samples from a likelihood model + excellent unsupervised representations
OpenAI
@OpenAI
Synthesizing realistic high-resolution images with Glow, a new reversible generative model: blog.openai.com/glow/
-
Ilya Sutskever
@ilyasut
a talk on some of our recent work: www.youtube.com/watch?v=9EN_HoEk3KY
-
Ilya Sutskever
@ilyasut
Compared to other society-altering sciences (physics, bio), ML is by far the easiest to get started with. Requirements: common sense, programming, and high frustration tolerance.
OpenAI
@OpenAI
Are you a programmer who wants to get into deep learning? Apply to our Machine Learning Fellow role: openai.com/jobs/
















































































































































































































