Qiayuan (@qiayuanliao) and I wrote about modeling the Unitree G1’s closed-loop ankle linkage in @mujoco.
We look at how the linkage changes the ankle’s motion and dynamics, and what it takes to simulate it reliably.
We spent quite a bit of time on the blog post making interactive demos and shareable models, and we hope you find it useful!
kzakka.com/closing-the-loop/
Pieter Abbeel
Pieter Abbeel (born 1977) is a professor of electrical engineering and computer sciences, Director of the Berkeley Robot Learning Lab, and co-director of the Berkeley AI Research (BAIR) Lab at the University of California, Berkeley. He is also the co-founder of Covariant, a venture-funded start-up that aims to teach robots new, complex skills, and co-founder of Gradescope, an online grading system that has been implemented in over 500 universities across the United States. He is best known for his cutting-edge research in robotics and machine learning, particularly in deep reinforcement learning. In 2021, he joined AIX Ventures as an Investment Partner. AIX Ventures is a venture capital fund that invests in artificial intelligence startups. More on Wikipedia
-
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
-
Pieter Abbeel reposted
Grounded Superintelligence
@GroundedSI
Introducing RoboCap-10k. 10,000 hours of open egocentric data with metric hand / head pose on every frame.
Captured on RoboCap across 11 countries and 21 industries.
GroundedAPI produces 3D enrichment giving robot policies high-fidelity action grounding.
with @BitRobotNetwork @huggingface @rerundotio ↓This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Qiayuan Liao
@qiayuanliao
SuperMimic is open-sourced during my talk at CORL.
Have fun with it: github.com/HybridRobotics/supermimic
Qiayuan Liao
@qiayuanliao
We found a way to scale humanoid control that reduces GPU-hours by more than 1000×.
Across 129,514 motions, SuperMimic surpasses the state of the art (128 GPUs, 7 days) with 1 GPU in 3 hours.
Project: supermimic.github.io
Demo: supermimic.github.io/demo/This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Jiaman Li
@jiaman01
Come to our VLK poster #189 today 5-6:30pm! Let’s discuss humanoid loco-manipulation! #corl2026
Jiaman Li
@jiaman01
🤖 How can we scale up humanoid robot learning?
Introducing 🌟VLK🌟: generating large-scale synthetic data with paired egocentric observations, text, and full-body G1 kinematics for learning humanoid loco-manipulation. No teleoperation needed!
Website: vision-language-kinematics.github.io -
Pieter Abbeel reposted
Grounded Superintelligence
@GroundedSI
We’re excited to announce RoboMoCap.
The largest egocentric + truly ground-truth whole-body dataset for humanoids.
We think this is an important first step towards bootstrapping whole-body generalization from scalable first-person POV data.
With @BitRobotNetwork @NokovMocap
Jiafei Duan✈️CoRL2026
@DJiafei
🚀 Introducing RoboMoCap!
We are releasing the largest locomotion-centric egocentric vision × optical motion capture dataset for humanoid robot learning, joint effort with @GroundedSI @BitRobotNetwork @NokovMocap
📊200+ hours of recordings
🏃 290+ diverse actions
👓 Egocentric Vision
🎥 120 Hz optical motion capture with ~10 mm tracking accuracy
🌐 bitrobot-foundation.github.io/robomocap/
🧵👇This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Qiayuan Liao
@qiayuanliao
Wanted to get into manipulation. Decided to first build a full stack that scales.
Plan: copy @GeneralistAI's gripper. Fun weekend project.
It was not.
So @t_k_233 and I documented every design choice and lesson along the way.
Meet Specialist: fully open-source hardware, data pipeline, training, and deployment code.
specialist-ai.github.ioThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Yilin Wu ✈️ CoRL 2026
@YilinWu11
I’m in Austin for @corl_conf to present three of my recent works, each with a workshop spotlight! 🤖
🧠 IMPRINT: RoboMem workshop
🤝 ViTaL: main conference + Learn@Deploy workshop
🔮 WEAVER: CSIR workshop
I’m also giving a joint keynote with @andrea_bajcsy at the Oopsie Data workshop and co-hosting Pretrain to Adapt workshop!
💬 We welcome your questions for our panel! Submit them on our workshop website:
🌐 corl2026-robotcl.github.io
Happy to catch up with old friends and make new ones if you are interested in those directions #corl2026 -
Pieter Abbeel reposted
Jitendra MALIK
@JitendraMalikCV
Robots that learn should generalize. Locomotion should show generalization across terrain; manipulation across object categories. Here is a video of “Pick Anything”, demonstrating human like grasps for 66 different objects, ranging in size and shape from air pods to dumbbells to pipe wrenches, using a *single* policy. The system uses the stereo camera head for vision (no wrist cameras) & Sharpa Wave hands for manipulation. It was trained using video and RL (zero teleoperation data). This is research from our Dex-Hands team at Amazon FAR and we will be doing live demos of the system at CoRL 2026 ( Demo 12, Nov. 9 &10). Please come by!
This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Kevin Frans
@kvfrans
Happy to release a small report on some studies at Cursor on stabilizing the combination of RL + self-distillation in production-like agentic settings. w/ @srush_nlp and @ashvinair
arxiv.org/abs/2611.05266
This work/setup is a bit stale now, but could provide some insight on what pitfalls to lookout for! -
Pieter Abbeel reposted
Kuba
@kuba_AI
AI is changing how we build software.
The next frontier is matter.
Meet CliqueFlowmer: upload a material, optimize its representation, generate a new candidate.
Our demo is live. Our paper is coming to #neurips2026 in #sydney.
Watch it happen. 🧵This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Xiaoyu Huang
@x_h_ucb
Robot control has seemed reserved for frontier LLMs like Astra and Opus 5.5. Our results challenge that view: robot-use is a general capability emerged across many LLMs, including open-weight models like Qwen and DeepSeek, without RSI, distillation, fine-tuning, or robot-specific data.
These open-weight models are already competitive when deployed out of the box: zero-shot Qwen3.8-Flash-Next scores 31.39 on RoboDojo, surpassing GPT-6 Astra’s result from official RoboDojo team. These capabilities also extend to real-world, where Qwen scores 53.50/100 across 19 real-world tasks, outperforming WAMs/VLAs such as Dream-1 and fine-tuned π-0.5.
Why LLMs can work as a robot policy? They approach much like solving a math problem: first understand the scene, then triangulate feature points across multiple views to locate them in 3D, and finally compute grasp poses using coordinate transforms and inverse kinematics. This process draws on four general capabilities: visual understanding, spatial grounding, coding in geometry, and embodied reasoning. Robot performance appears to emerge once these capabilities are sufficiently strong.
This computation-heavy approach can yield sub-millimeter precision when feature points are correctly identified. This suggests that improvements in Robot-Use Agents should instead focus on high-level intents and visual grounding, including strategy, object affordances, 2D feature understanding and pinpointing, rather than on low-level actions.
With open-weight models, we can now train, test, and improve these capabilities directly.
Blog: openrua.ai/uni-robot-agent
Code: github.com/Open-RUA/uni-robot-agent
Paper: arxiv.org/abs/2611.01560 -
Pieter Abbeel reposted
Yuheng Qiu
@QiuYuhengQiu
🤖 𝐒𝐒𝐈𝐎: 𝐒𝐞𝐥𝐟-𝐒𝐮𝐩𝐞𝐫𝐯𝐢𝐬𝐞𝐝 𝐈𝐧𝐞𝐫𝐭𝐢𝐚𝐥 𝐎𝐝𝐨𝐦𝐞𝐭𝐫𝐲
Since the first learned inertial odometry 10 years ago, every method has needed ground-truth labels.
It doesn't have to: SSIO learns from raw IMU, fully self-supervised 🧵👇
ssio-imu.github.io -
Pieter Abbeel reposted
Amber Xie
@amberxie_
Collecting data for dexterous hands is hard. Teleop needs a full setup. Bringing UMI-like devices to 20-dof hands is challenging.
Introducing DexPort 🖐️: a hand-held interface w/ a real robot hand. No embodiment gap, recorded joint state AND actions, easy to use.This media couldn't be displayed. See it on the original post. -
Pieter Abbeel
@pabbeel
We 'll be hiring very actively at Berkeley!
Assistant/ Associate/ Full Professor – Electrical Engineering & Computer Sciences (Deadline: Dec. 7, 2026 at 11:59 pm Pacific Time). Apply here: aprecruit.berkeley.edu/JPF05591
Assistant Professor – AI for Biology – Electrical Engineering & Computer Sciences and Center for Computational Biology (Deadline: Nov. 9, 2026 at 11:59 pm Pacific Time) Apply here: aprecruit.berkeley.edu/JPF05560
Assistant Professor – Intersection of Statistics and CS – Departments of Electrical Engineering & Computer Sciences and Statistics (Deadline: Nov. 10, 2026 at 11:59 pm Pacific Time). Apply here: aprecruit.berkeley.edu/JPF0556
Assistant/ Associate/ Full Professor – Computational Biomedicine - Center for Computational Biology (Deadline: Nov. 18 2026 at 11:59 pm Pacific Time). Apply here: aprecruit.berkeley.edu/JPF05568
Assistant / Associate Professor – Electronic, Functional, and Quantum Materials – Electrical Engineering & Computer Sciences and Materials Science & Engineering (Deadline: Dec. 7, 2026 at 11:59 pm Pacific Time). Apply here: aprecruit.berkeley.edu/JPF05584
Coming Soon! Assistant/ Associate/ Full Professor - AI and the Physical Sciences -Bakar Institute of Digital Materials for the Planet (BIDMaP) and home department(s) to be determined. (Deadline: Dec. 7, 2026 at 11:59 pm Pacific Time) Apply here: aprecruit.berkeley.edu/JPF05619
Coming Soon! Assistant/ Associate/ Full Professor-in-Residence - Gladstone Institute of Data Science and Biotechnology and home department(s) to be (Deadline: TBD at 11:59 pm Pacific Time) Apply here: aprecruit.berkeley.edu/JPF05621 -
Pieter Abbeel reposted
C. Zhang
@ChongZzZhang
It is a new paradigm -- world model + APG. This enables 1000x faster training of generalist whole-body tracker. And opus 5.5 high can reproduce it within 1h for lafan.
Recently I wonder if we are turning dumb and falling into scaling slop. We need more @qiayuanliao .
Qiayuan Liao
@qiayuanliao
We found a way to scale humanoid control that reduces GPU-hours by more than 1000×.
Across 129,514 motions, SuperMimic surpasses the state of the art (128 GPUs, 7 days) with 1 GPU in 3 hours.
Project: supermimic.github.io
Demo: supermimic.github.io/demo/This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Philipp Wu
@philippswu
While frontier models have show some impressive zero shot demos for robotics, there remains a wide specrum of physical capabilities where there remains a gap. Our goal with benchmarks.xdof.ai is to better understand these models in a target manner.
Stay tuned for more!
XDOF
@xdofai
Announcing: the XDOF Robotics Leaderboard
Rigorous evaluations on real robots are an important and overlooked ingredient in solving general purpose robotics.
Our goal is to provide trusted evaluations, with meaningful benchmark design and thorough rollout execution that researchers can rely on.
The Robotics Leaderboard is where we’ll publish how frontier models perform against our benchmarks. We’ll update the leaderboard with every model we evaluate. -
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
A nice example of physics sims and learned world models working together! The simulator provides accurate transitions and the world model amortizes them into a much faster, differentiable surrogate for policy optimization. Play with it in your browser: supermimic.github.io/demo/
Qiayuan Liao
@qiayuanliao
We found a way to scale humanoid control that reduces GPU-hours by more than 1000×.
Across 129,514 motions, SuperMimic surpasses the state of the art (128 GPUs, 7 days) with 1 GPU in 3 hours.
Project: supermimic.github.io
Demo: supermimic.github.io/demo/This media couldn't be displayed. See it on the original post. -
Pieter Abbeel
@pabbeel
Game changer for humanoid full-body control training. Now it's superfast (and cheap, just one GPU) to train a single policy on *all* the motions. How come:
(1) World model much faster than regular sim
(2) Backprop through time collects way more signal than policy gradients
Qiayuan Liao
@qiayuanliao
We found a way to scale humanoid control that reduces GPU-hours by more than 1000×.
Across 129,514 motions, SuperMimic surpasses the state of the art (128 GPUs, 7 days) with 1 GPU in 3 hours.
Project: supermimic.github.io
Demo: supermimic.github.io/demo/This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
zeji yi
@ZejiYi
Common belief: Robot policies need continuous action heads; discrete actions are 𝐥𝐞𝐬𝐬 𝐧𝐚𝐭𝐢𝐯𝐞 𝐚𝐧𝐝 𝐥𝐞𝐬𝐬 𝐩𝐫𝐞𝐜𝐢𝐬𝐞.
❗We find policies w/ the simplest discrete representations 𝐦𝐚𝐭𝐜𝐡 𝐨𝐫 𝐛𝐞𝐚𝐭 continuous ones—in sim & real, in single-step & diffusion/flow settings.
Here's why 🧵 -
Pieter Abbeel
@pabbeel
Congratulations @pulkitology @haarnoja and team, very excited to see what's ahead!
Pulkit Agrawal
@pulkitology
@EkaRobotics has raised a $150M Series A.
Generality v/s Performance is the fundamental tension at the frontier of robotics. Generality without performance is just a prototype. Performance without generality is just a single-purpose tool.
Customers shouldn’t have to choose. Eka breaks the tradeoff. Powered by our Vision-Force-Action (VFA) model, we deliver the performance and versatility customers need to put robots to work every day.
Eka is built on three convictions:
⚡️Force is the language of manipulation.
🚀 Reinforcement learning is how dexterous manipulation scales.
🎂 System 1—fast, reliable physical execution—is the bottleneck to solve.
With this funding, we will scale our model and accelerate customer deployments.
Grateful to be building alongside a dream team and investors who share our ambition.This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Carlo Sferrazza
@carlo_sferrazza
Two nice findings in this study on touch-driven curiosity:
1. RL from scratch: with no task reward, only agents that are curious about touch learn to grasp. That holds even though every variant observes touch, so the difference comes from the curiosity component. And the data they collect is so interaction-rich that it can be used for fully offline learning of pick-and-place.
2. VLA post-training: when steering a frozen VLA that never saw touch, adding touch as an observation helps, but the best results come from also making the robot curious about touch, especially on the harder, contact-rich tasks.
Check out the work in Klemens' post below 👇
Klemens Iten
@klemensiten
Excited to share that our paper «Tactile Curiosity Drives Robot Interaction» has been accepted to #corl2026!
We study how tactile feedback can help robots explore more efficiently. Great collaboration between @CSatETH @berkeley_ai @ETH_AI_Center
Website: kiten.me/projects/tacex -
Pieter Abbeel reposted
XDOF
@xdofai
We heard the hype around LLMs and robotics, so we put Astra and Fable to the test.
16 tasks testing 13 capabilities.
Our finding: both did surprisingly well on reasoning and planning and struggled with dexterity. But Astra did better overall.
Here’s what happened: -
Pieter Abbeel reposted
Dylan Foster 🐢
@canondetortugas
A super exciting announcement: In January 2027, I'll be joining @UCBerkeley as an assistant professor in EECS and Statistics.
I'll be recruiting PhD students and postdocs this cycle, starting Fall 2027. If you're interested the science of RL and deep learning, please apply.This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Yilin Wu ✈️ CoRL 2026
@YilinWu11
What to remember in the past to act—spatial, temporal or motion details?
We often assume what’s useful, but that’s hard to know in advance.
Meet IMPRINT 🧠: retain rich memory first, then learn to use it for control. Compact latents keep inference fast as history grows. 🧵 1/10This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Zhen Wu
@zhenkirito123
Perceptive Humanoid Parkour (PHP) is now OPEN-SOURCE! 🔥
Try training your own parkour robot! 🤖
🔗 Project: php-parkour.github.io
💻 Code: github.com/amazon-far/php_parkour
🎮 Try it in your browser: php-parkour.github.io/demo.html
Zhen Wu
@zhenkirito123
Can humanoids perform agile, autonomous, long-horizon parkour—based on what they see in the world?
We present 𝗣𝗲𝗿𝗰𝗲𝗽𝘁𝗶𝘃𝗲 𝗛𝘂𝗺𝗮𝗻𝗼𝗶𝗱 𝗣𝗮𝗿𝗸𝗼𝘂𝗿 (𝗣𝗛𝗣): a framework that chains dynamic human skills using onboard depth perception for long-horizon traversal.
1/6 -
Pieter Abbeel reposted
Yen-Jen Wang ✈️ CoRL
@wangyenjen
Practice makes perfect. A truly agentic robot shouldn’t just know what to do. It should also figure out how to get better.
What if a robot could watch a demonstration, figure out what it needs to practice, and learn how to get better through practice?
Introducing RPG: Guided Self-Improvement for Embodied Agents, a real-to-sim-to-real pipeline that turns real-world demonstrations into targeted practice.
RPG reconstructs practice tasks from demonstrations, uses demo videos to guide skill improvement, and continually updates a shared skill system from failures.
On real robots, RPG achieves 30/30 successes across 3 tasks. In simulation, it reaches 95.0% success across 22 manipulation tasks after 15 rounds of self-improvement.
Demo → Reconstruct → Guided Practice → Self-Improve → Go Real -
Pieter Abbeel reposted
Grounded Superintelligence
@GroundedSI
Today, we are announcing GroundedSLAM.
LaMAria is the most difficult egocentric SLAM benchmark to date. We are the first to close the gap on Aria’s SLAM and eclipse all other methods by a wide margin.
SLAM is the foundation of all 3D estimation. We think this is a big unlock for better robotics 👇🧵This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Songbo Hu
@songbo_
Can a humanoid learn robust whole-body loco-manipulation directly from a human? No robot, no teleop, no retargeting.
Meet Workhorse 🐴: our G1 topples and climbs a suitcase, catches a thrown box, and sorts boxes with hands and a kick. Fully autonomous.
More videos atThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Qiayuan Liao
@qiayuanliao
Meet Workhorse 🐴: we study how to learn whole-body loco-manipulation skills from human demos without teleop.
Here is my favourite video: we tried everything to make it fail🏒.
One take, real-time, fully autonomous.This media couldn't be displayed. See it on the original post.
Songbo Hu
@songbo_
Can a humanoid learn robust whole-body loco-manipulation directly from a human? No robot, no teleop, no retargeting.
Meet Workhorse 🐴: our G1 topples and climbs a suitcase, catches a thrown box, and sorts boxes with hands and a kick. Fully autonomous.
More videos atThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Chung Min Kim
@ChungMinKim
Excited to introduce QF3: Fast Flow RL with Filtered Q-Gradients!
Train humanoids from scratch, fine-tune VLAs, even fine-tune image models... all with one simple off-policy update ✨ -
Pieter Abbeel reposted
Zihan Wang
@Z1hanW
Humans generalize from rich interaction experience. We bring this to humanoid loco-manipulation through scalable real-to-sim-to-real.
Introducing PRISM, we expand 4 real videos into 256 counterfactual variants, recover robot-object trajectories, and train one policy that generalizes across diverse objects, scales, and spatial layouts.
CoRL’26 🚀 Website for details 👉 prism-real2sim2real.github.io
Work done at Amazon FAR w/ @zhenkirito123 @pabbeel @rocky_duan @JitendraMalikCV @carlo_sferrazza @ckarenliu @GuanyaShi @akanazawaThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Arthur Allshire
@arthurallshire
check out our sim and code release! some super useful envs for evals, rl etc for people to try out!
Adam Rashid
@adamrasb
Happy to announce the full ABC release! We’re also excited that ABC was accepted to CoRL 2026!
Check out our website for code, 400+ hours of sim data on 24 tasks, and 5,850 labeled policy-evaluation episodes.
@arthurallshire @Cinnabar233 @ritvik_singh9 @redstone_hong @davidrmcallThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Hongsuk Benjamin Choi
@redstone_hong
ABC is fully out and headed to CoRL 2026!
ABC-130K is the largest open teleop dataset to date: 3,500 hours, 130K+ episodes, 195 tasks, collected on an $8K bimanual setup. The full stack is open: hardware, training code, sim, and eval, so academic labs can do this research on equal footing.
Beyond the data, we ran the baseline science for the community to stack on:
• Sim-to-real correlation (r = 0.91 on task progress)
• Which offline metrics matter?
• Scaling law in compute (training steps and batchsize)
• Conditioning the policy
Project page: abc.bot
Github: github.com/amazon-far/abc
Adam Rashid
@adamrasb
Happy to announce the full ABC release! We’re also excited that ABC was accepted to CoRL 2026!
Check out our website for code, 400+ hours of sim data on 24 tasks, and 5,850 labeled policy-evaluation episodes.
@arthurallshire @Cinnabar233 @ritvik_singh9 @redstone_hong @davidrmcallThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Adam Rashid
@adamrasb
Happy to announce the full ABC release! We’re also excited that ABC was accepted to CoRL 2026!
Check out our website for code, 400+ hours of sim data on 24 tasks, and 5,850 labeled policy-evaluation episodes.
@arthurallshire @Cinnabar233 @ritvik_singh9 @redstone_hong @davidrmcallThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Lerrel Pinto
@LerrelPinto
It's clear that we’re heading toward an abundance of human data. Great to see @GroundedSI pushing forward on the infra layer for it.
What’s less clear is how we’ll turn that into an abundance of robots. The next few months should be fun 😉
Grounded Superintelligence
@GroundedSI
Grounded API is live.
- SOTA on hand-tracking benchmarks (< 1 cm)
- SOTA on SLAM benchmarks
- In-the-wild ego data -> enriched data in minutes
- Integration with @huggingface @LeRobotHF & @rerundotio
- Built for @BitRobotNetwork RoboCap suite
Technical report & more↓This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Grounded Superintelligence
@GroundedSI
Grounded API is live.
- SOTA on hand-tracking benchmarks (< 1 cm)
- SOTA on SLAM benchmarks
- In-the-wild ego data -> enriched data in minutes
- Integration with @huggingface @LeRobotHF & @rerundotio
- Built for @BitRobotNetwork RoboCap suite
Technical report & more↓This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Ademi Adeniji
@AdemiAdeniji
This has been a long time coming.
Announcing GSI and our first release: the first full-stack product for state-of-the-art robotics data. Built by roboticists, for robotics.
Starting today, you can get a lightweight 6 camera cap and wrist cameras: fully wireless, ergonomic, and sensor complete.
From there, Grounded API produces SOTA 3D enrichments at the click of a button.
From meeting my co-founder @vincentjliu during my first year at Stanford, we’ve been waiting for this moment in AI.
Fast forward through my PhD with @pabbeel at UC Berkeley and our work on EgoZero and FeeltheForce. We've come full circle.
Immense gratitude to our early investors @pabbeel @naval @LerrelPinto @Thom_Wolf @_milankovac_ @rob_fergus and many others.
All success in robotics comes down to leveraging the best data. Today, that is where we start.
Grounded Superintelligence
@GroundedSI
Grounded API is live.
- SOTA on hand-tracking benchmarks (< 1 cm)
- SOTA on SLAM benchmarks
- In-the-wild ego data -> enriched data in minutes
- Integration with @huggingface @LeRobotHF & @rerundotio
- Built for @BitRobotNetwork RoboCap suite
Technical report & more↓This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Siting Li
@SitingLi627
1/9 Image Tokenizers define Visual Languages. 🎨🔤
Image tokenizers do more than compress and reconstruct pixels. In unified multimodal models, they define the visual language a single model must learn, align with text, and use for both understanding and generation — and the best compressor may not produce the easiest language to learn.
We study image tokenizers through the lens of unified AR multimodal training. 🧵 -
Pieter Abbeel reposted
Peter Reinhardt
@reinpk
Announcing @Revoy’s $27M Series A
Diesel is expensive. But long-haul EV semis are even more expensive! Today we’re launching the United States’ first hybrid electric, cost-competitive long-haul freight network.
You don’t see it on your credit card statement, but the average American spends $1,270/year on long-haul freight. Diesel prices at the pump are outrageous today: rising on both international price shocks and increasing domestic shale oil costs. And pure EV semis are somehow even more expensive to operate. We need a better solution, and Ian Rust @semiautonomy figured it out. Revoy’s unique powered converter dolly lets us drop diesel usage by 95% and efficiently replace it with low-cost electricity.
After two years of demonstrations and iterative improvements, we’re now taking the unique Revoy vehicle that we design and manufacture, and launching our hybrid-electric freight network with a select set of carriers. Shippers can reduce diesel consumption immediately on Revoy’s network without having to convince legacy carriers to convert their fleets, and without any investment in new infrastructure or equipment.
We’re starting in the Pacific Northwest, with additional lanes planned. -
Pieter Abbeel reposted
Zongheng Yang
@zongheng_yang
Today, SkyPilot is out of stealth.
Building custom intelligence is now existential. We help frontier AI teams build intelligence faster by removing their biggest bottleneck: AI compute fragmentation.
Frontier teams like @appliedcompute, @AbridgeHQ, @hippocraticai, @hcompany_ai, and @nubank already run on SkyPilot, with 10x faster time-to-intelligence and double-digit increase in GPU utilization.
AI teams today get compute anywhere they can. They then firefight compute fragmentation across providers. Researchers burn time on workload setup. Infra gets paged when GPUs go down. Frontier teams build slowly even on the fastest compute.
@skypilot_org turns your fragmented compute into one AI supercomputer, so you run frontier workloads faster. Many users manage 10,000+ GPUs across providers with SkyPilot. GPU hours consumption has grown 6x in the last 6 months.
1/ We're launching SkyPilot Platform — the AI compute platform for frontier AI teams to manage large GPU fleets and accelerate building custom intelligence.
Optimized for fleet management, team governance, and frontier workloads — pretraining, post-training, multi-cluster serving, and sandboxes. SkyPilot open source users can switch to the platform with a server URL change.
2/ We've raised over $20M led by @Lux_Capital (@breeves08), with participation from @AmplifyPartners (@dauber, @lennypruss), @coatuemgmt, @FoundationCap (@ashugarg , @JayaGup10), @RaceCapital, @thehousefund, and top operators like @alighodsi (CEO, Databricks), @JeffDean (Chief Scientist, Google), @rauchg (CEO, Vercel), @amasad (CEO, Replit), @ClemDelangue (CEO, @huggingface) and more.
We're hiring across Engineering and GTM to deliver the platform for the next decade of AI.
Above all, I'm excited to be building with the incredible team we've assembled, along with my cofounders Zhanghao @Michaelvll1, Romil @bromil101, Scott, and Ion @istoica05.
If you firefight AI compute, let's build. -
Pieter Abbeel
@pabbeel
Check out ABC Box: i2rt offers a robot setup nicely compatible with ABC data/training infra, even easier to get started!
i2rt.com/products/abc-box
Ritvik Singh
@ritvik_singh9
Introducing ABC: open data, training, and infrastructure for robotics.
We release the largest teleop dataset to date, and extensively investigate design decisions, pretraining, and post-training techniques.
@arthurallshire @Cinnabar233 @adamrasb @redstone_hong @davidrmcallThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Yen-Jen Wang ✈️ CoRL
@wangyenjen
How can we scale perception-based humanoid learning without collecting massive humanoid teleoperation data?
🚀 Excited to finally share VLK!
What excites me most about VLK is that it reframes data collection as a data generation problem. Instead of relying on expensive humanoid teleoperation, we automatically generate synchronized vision, language, and whole-body kinematics from reconstructed real-world scenes.
Making this vision a reality required bridging three fundamental challenges:
👀 Perception: Bridging the RGB sim → real gap through visual domain randomization and motion blur mitigation during both training and deployment.
🤖 Embodiment: Bridging the kinematics → dynamics gap with real-time VLA deployment, test-time RTC, and SceneBot, enabling seamless deployment on a real humanoid.
🌍 Environment: Bridging the real-world → synthetic gap to enable scalable Vision-Language-Kinematics data generation through scene reconstruction and interaction synthesis.
It has been an amazing journey working with such an incredible team. For a complete walkthrough of the project, check out @jiaman01's thread below 👇
🌐 Project: vision-language-kinematics.github.io
📄 Paper: arxiv.org/abs/2606.30645
🎦 Video: youtu.be/ZB6k_iMJP7M
Huge thanks to my amazing collaborators @jiaman01 @eric_srchen @TakaraTruong @ Pei Xu, and to our advisors @pabbeel @rocky_duan @KoushilSreenath @akanazawa @carlo_sferrazza @GuanyaShi @ckarenliu.
Jiaman Li
@jiaman01
🤖 How can we scale up humanoid robot learning?
Introducing 🌟VLK🌟: generating large-scale synthetic data with paired egocentric observations, text, and full-body G1 kinematics for learning humanoid loco-manipulation. No teleoperation needed!
Website: vision-language-kinematics.github.io -
Pieter Abbeel reposted
Jiaman Li
@jiaman01
🤖 How can we scale up humanoid robot learning?
Introducing 🌟VLK🌟: generating large-scale synthetic data with paired egocentric observations, text, and full-body G1 kinematics for learning humanoid loco-manipulation. No teleoperation needed!
Website: vision-language-kinematics.github.io -
Pieter Abbeel reposted
Brian Smith
@sirwart
There's a lot to like about this paper, but my favorite part is the data format used for training. I adopted it and it's a big improvement over what I was doing before. If you want to try it out, I pulled it out into it's own library here: github.com/tetra-dynamics/srtf
Ritvik Singh
@ritvik_singh9
Introducing ABC: open data, training, and infrastructure for robotics.
We release the largest teleop dataset to date, and extensively investigate design decisions, pretraining, and post-training techniques.
@arthurallshire @Cinnabar233 @adamrasb @redstone_hong @davidrmcallThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Philipp Wu
@philippswu
Data quality is too often the unsung hero for deep learning. Excited to finally share WARP-RM, where we train a model to learn the rate of task progress, rather than absolute progress, by warping the timescale of existing trajectory data. Check out @uynitsuj 's thread for more! 👇
Justin Yu ✈️ CoRL
@uynitsuj
Introducing WARP-RM: A Warp-Augmented Relative Progress Reward Model for Data Curation 🧵
We gave our t-shirt folding robot more demonstrations and it got worse. Every extra demo ended in a successfully folded shirt. The data wasn't bad. It was noisy. The policy couldn't tell productive motion from dead time, and it imitated both equally. So which moments of a demo are actually worth copying?
🌐 Project Website: uynitsuj.github.io/warp-rm
📄 Paper: arxiv.org/abs/2606.28320
💻 Code: github.com/uynitsuj/WARP-RM
📨 XDOF blog post: xdof.ai/blog/warp-rmThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Etched
@Etched
We're coming out of stealth.
We've built our first racks after a successful A0 tapeout, $1B+ in customer contracts, and $800m raised.
Early customer tests show us achieving SOTA throughput, latency, and power efficiency on inference workloads.
Our first racks ship this summer. -
Pieter Abbeel reposted
Jessy Lin
@realJessyLin
we started a company!!
so, we’re tackling continual learning: what’s the learning algorithm to take arbitrary data — documents, conversations, the models’ own experience — and make better models? how do we scale compute in the same way we’ve already seen with pre-training and inference time, but scaling on the same data we see as humans, day after day with no labels, no rewards?
A lot of the ingredients are out there already (rl, distillation, long-context, sparse / param-efficient architectures, etc.). our team is at the frontier of these topics, and we’re singularly focused on this. we want to understand this problem better than anyone else in the world.
nobody’s solved this problem yet, but even today it’s extremely greenfield opportunity to co-develop research & useful products. in our space, how people interact with the models defines what the data distribution is - and working on this problem end-to-end, from core science to end user, gives us incredible freedom to define the problem and imagine new kinds of experiences.
i expect we’ll use models that continually learn much differently than we’re using them today. it’ll feel different when the models _just know_, and build on our thinking and direction in ways we can’t even imagine. we don’t even know the queries we’re not asking, the things we would do but aren’t able to today.
i’m so excited to share what we’re doing with the world in the coming months!!
and the team is extremely cracked :) tackling this grand challenge and working alongside @jxmnop @EyubogluSabri @dan_biderman @MayeeChen @__howardchen @shizhehe and many others has made every day so fun. come work with us! -
Pieter Abbeel reposted
Mahi Shafiullah 🛫 CoRL 🏠🤖
@notmahi
Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data?
Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Will Liang
@willjhliang
Introducing Do as I Do 👀, a framework to transform everyday human videos into 100s of dexterous robot demos. Co-led with @bhawna_paliwal_ and @HarithejaE, and check out @notmahi's thread!
Here’s a little preview of our dexterous manipulation results. More about how we produce them from human reconstructions in this mini-thread! 🧵
Mahi Shafiullah 🛫 CoRL 🏠🤖
@notmahi
Robots are the bottleneck in scaling robotics, and learning from human video promises to solve it. But how can chaotic human data ever measure up to sanitized, lab-made teleoperation data?
Introducing Do as I Do: establishing a much needed correspondence between human videos and dexterous robot data. Some fun insights below: 🧵This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
XDOF
@xdofai
We are proud to release ABC-130K, the largest open bimanual manipulation dataset.
ABC-130K includes 130K+ trajectories covering 200 complex manipulation tasks. All are performed on a low-cost bimanual rig (YAM by I2RT).
The data is available under the highly permissive Apache 2.0 license. -
Pieter Abbeel reposted
Adam Rashid
@adamrasb
Super excited to be able to release this project we've been working on. The largest open robotics dataset to date and policies capable of dexterous manipulation!
This media couldn't be displayed. See it on the original post.
Ritvik Singh
@ritvik_singh9
Introducing ABC: open data, training, and infrastructure for robotics.
We release the largest teleop dataset to date, and extensively investigate design decisions, pretraining, and post-training techniques.
@arthurallshire @Cinnabar233 @adamrasb @redstone_hong @davidrmcallThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Haozhe Jiang
@erichzjiang
Why aren’t Diffusion Language Model smart yet? Lacking stable post training is a major bottleneck!
Meet DiPOD: the tripod for diffusion model post-training.
DiPOD boosts accuracy across reasoning tasks, with Sudoku jumping from 22% to 97%, through a one-line code change.
🧵1/5 -
Pieter Abbeel reposted
Ritvik Singh
@ritvik_singh9
Introducing ABC: open data, training, and infrastructure for robotics.
We release the largest teleop dataset to date, and extensively investigate design decisions, pretraining, and post-training techniques.
@arthurallshire @Cinnabar233 @adamrasb @redstone_hong @davidrmcallThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Philipp Wu
@philippswu
ABC-130K is the largest open bimanual manipulation dataset. It combines dataset scale (4x larger than the next YAM dataset) and quality on a low-cost embodiment (YAM from I2RT).
This size and depth allows researchers to conduct a wider range of experiments and achieve better policy results.
The dataset is released under the highly permissive Apache 2.0 license and can be accessed below. Let us know what you think!
xdof.ai/blog/abc-130k -
Pieter Abbeel reposted
Philipp Wu
@philippswu
Our collaborators published ABC: Scalable Behavior Cloning with Open Data, Training, and Evaluation using the ABC-130K dataset and XDOF services.
abc.bot
Thank you to UC Berkeley lead authors on this work:
Arthur Allshire @arthurallshire
Himanshu Gaurav Singh @Cinnabar233
Ritvik Singh @ritvik_singh9
Adam Rashid @adamrasb
Hongsuk Choi @redstone_hong
David McAllister @davidrmcall
And collaborators:
Pieter Abbeel @pabbeel
Peter Chen @peterxichen
Rocky Duan @rocky_duan
Phillip Isola @phillip_isola
Jitendra Malik @JitendraMalikCV
Guanya Shi @GuanyaShi
Angjoo Kanazawa @akanazawa -
Pieter Abbeel reposted
Philipp Wu
@philippswu
Announcing @xdofai: We’ve raised $70 million to build the core robotic infrastructure ecosystem for robot foundation models.
My cofounders Fred (@YideShentu), Nemo (@itsnemojin) and I have been pursuing the dream of general purpose robots for our entire lives. After work at Covariant, Meta and Tesla, it became clear to us that general purpose robots are coming, and we are building XDOF to help make them a reality.
For the last two years, we’ve been working behind the scenes to support major labs and companies deploying robots. In us, they have a partner with full-stack expertise, from hardware to operations to policy training.
As our first public contribution to the space, we are open-sourcing ABC-130K, the largest open source teleoperation dataset, in collaboration with our partners from UC Berkeley, Carnegie Mellon, MIT and Amazon FAR.
Thank you to our customers, partners, collaborators and investors for your trust and conviction in us. Together, we can accelerate the future of robotics! -
Pieter Abbeel reposted
Siheng Zhao✈️CoRL
@SihengZhao
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇 -
Pieter Abbeel reposted
Guanya Shi
@GuanyaShi
Best Conference Paper Award and the Best Paper Award on Robot Manipulation and Locomotion at #icra2026!
Guanya Shi
@GuanyaShi
High-quality motion reference data is key for humanoid skill learning 🤖🕺💃
A natural idea is to leverage human motions and “translate” them to humanoid motions, a process known as retargeting. For interaction-rich tasks such as scene interaction and loco-manipulation, retargeting is challenging: it must ensure motion consistency, smoothness, kinematic feasibility (no artifacts like penetration or foot skating), and scalability (one framework can handle thousands of motions).
Excited to release OmniRetarget — a scalable retargeting method with a 4-hour high-quality humanoid motion dataset for interaction-rich tasks. OmniRetarget takes an interaction-preserving perspective: we optimize Laplacian deformation between source and target interaction meshes while enforcing kinematic constraints, producing consistent, smooth, and feasible trajectories at scale. Even better, OmniRetarget can efficiently augment motions by varying terrains, objects, and initial poses.
This high-quality interaction-preserving retargeting enables a minimal RL setup to execute long-horizon (up to 30s) agile, interaction-rich skills. All tasks in the video share just 5 rewards, 4 domain randomization terms, and rely only on proprioception.
More details: omniretarget.github.io
Zhen Wu
@zhenkirito123
Humanoid motion tracking performance is greatly determined by retargeting quality!
Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL:
- 5 rewards,
- 4 DR terms,
- Proprio. ONLY,
- NO history/curriculum.
Ready for agile, human-like 🤖? (Best with 🎧)
🔗 omniretarget.github.io 🎥
1/9This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Pulkit Agrawal
@pulkitology
Eka means unity -- “one,” in Sanskrit and “first” in Finnish.
We’re building intelligence for the physical world in its native language: forces.
Until now, robotics faced a tradeoff — generality or speed. The real world requires both. Robotics also faced a data problem.
Our Vision–Force–Action (VFA) model — the first of its kind — breaks the generality-speed tradeoff and the data barrier.
It's a new foundation uniting performance, generality, and safety for putting capable robots in everyone's hands.
Today, I am excited to share our journey of pushing robots beyond human limits.
Today, dexterity becomes scalable.
Today, I welcome you to the Era of Eka.
Co-founded with @haarnoja, and so thrilled and grateful to be working with a dream team at @EkaRobotics.
Learn more: www.ekarobotics.com -
Pieter Abbeel reposted
Kuba
@kuba_AI
Optimize materials yourself! 🔥
No cloning, no install — just open our interactive Google Colab demo and start optimizing crystals for formation energy or band gap in < 2 minutes.
→ Try it here: colab.research.google.com/drive/1usUg7zezFkcYHlm2MdYwZUNJ…
→ Or grab the repo + open weights: github.com/znowu/CliqueFlowmer huggingface.co/iamkuba/CliqueFlowmer -
Pieter Abbeel reposted
Guanya Shi
@GuanyaShi
I’m so tired of writing rebuttals to this kind of “lack of novelty” review: “This paper trivially combines A, B, and C, so the algorithmic novelty is limited.”
Technically, most (if not all) robotics papers are convex combinations of existing ideas.
I still deeply appreciate A+B+C papers—especially when they deliver:
- New capabilities: the “trivial combination” unlocks behaviors we simply couldn’t achieve before
- Sensible & organic design: A+B+C is clearly the right composition—not some arbitrary A′+B+C′
- Nontrivial interactions: careful analysis of the dynamics, coupling, or failure modes between A, B, C
- Rehabilitating old ideas: A was dismissed for years, but paired with modern B/C, it suddenly works—and teaches us why
- System-level & "interface" insight: the contribution is not any single piece, but how the pieces talk to each other
- Scaling laws or regimes: identifying when/why A+B+C works (and when it doesn’t)
- Engineering clarity: making something actually work robustly in the real world is not “trivial”
- New problem formulations: sometimes the real novelty is in the reformulation—only under this view does A+B+C make sense.
Maybe worth keeping these in mind when reviewing the next A+B+C paper : ) -
Pieter Abbeel reposted
Kuba
@kuba_AI
AI can optimize materials 🤘
Our (@pabbeel, @svlevine, @AIatMeta) proposed transformer model 𝗖𝗹𝗶𝗾𝘂𝗲𝗙𝗹𝗼𝘄𝗺𝗲𝗿, combined with 𝗲𝘃𝗼𝗹𝘂𝘁𝗶𝗼𝗻 strategies, discovers materials that optimize target properties.
arxiv.org/abs/2603.06082 -
Pieter Abbeel reposted
Guangqi Jiang
@LuccaChiang
Ever want to have a single policy to control diverse robots as well as different dexterous hands, or to observe the emergent behavior under cross embodiment training?
Introducing our #cvpr2026 paper XL-VLA, Cross-Hand Latent Representation for Vision-Language-Action Models. -
Pieter Abbeel reposted
Chen Feng
@simbaforrest
🤖We are hiring multiple Summer'26 Research Interns at @amazon FAR to work on open-world navigation and robot foundation models, especially in neural rendering & simulation/predictive world models/reasoning & agency/real-world evaluation/long-term autonomy!
-
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
Happy Friday!! mjlab v1.2.0 is out. This is our biggest release yet with 60+ PRs from 12 contributors.
pip install mjlab
Some highlights include:
- New more powerful domain randomization module
- Revamped ergonomic viewers
- Cloud training via @SkyPilot
- Complete doc rewrite -
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
mjlab now supports cloud training via SkyPilot. One command launches a GPU instance, syncs your code, trains, and tears down when done. We support 2 modes: direct uv install and Docker. Multi-GPU and hyperparameter sweeps work out of the box.
mujocolab.github.io/mjlab/main/source/training/cloud.html -
Pieter Abbeel reposted
Will Liang
@willjhliang
Introducing DreamDojo, a generalist, interactive robot world model pretrained on 44k hours of human egocentric videos!
Scaling human data pretraining with the largest and most diverse dataset to date for robot world model learning, we achieve spectacular generalization to unseen object interactions and environments. Additionally, after distillation, DreamDojo achieves real-time generation at 10 FPS—you can watch it predict the future as actions stream online, one frame at a time! With these capabilities, we show several key applications: live teloperation, policy evaluation, and model-based planning.
See our website: dreamdojo-world.github.io
As well as our paper: arxiv.org/abs/2602.06949
And github: github.com/NVIDIA/DreamDojo
And a quick dive with me… 🧵This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Zhen Wu
@zhenkirito123
Can humanoids perform agile, autonomous, long-horizon parkour—based on what they see in the world?
We present 𝗣𝗲𝗿𝗰𝗲𝗽𝘁𝗶𝘃𝗲 𝗛𝘂𝗺𝗮𝗻𝗼𝗶𝗱 𝗣𝗮𝗿𝗸𝗼𝘂𝗿 (𝗣𝗛𝗣): a framework that chains dynamic human skills using onboard depth perception for long-horizon traversal.
1/6 -
Pieter Abbeel reposted
Eric Jang
@ericjang11
Incredibly impressive. We are in the early days of seeing how much there is yet to be unlocked by the humanoid form factor
Zhen Wu
@zhenkirito123
Can humanoids perform agile, autonomous, long-horizon parkour—based on what they see in the world?
We present 𝗣𝗲𝗿𝗰𝗲𝗽𝘁𝗶𝘃𝗲 𝗛𝘂𝗺𝗮𝗻𝗼𝗶𝗱 𝗣𝗮𝗿𝗸𝗼𝘂𝗿 (𝗣𝗛𝗣): a framework that chains dynamic human skills using onboard depth perception for long-horizon traversal.
1/6 -
Pieter Abbeel reposted
Carlo Sferrazza
@carlo_sferrazza
Check out our latest work at FAR: Perceptive Humanoid Parkour. We show how to traverse challenging terrains with whole-body coordination, based on online depth perception! Read the details in @zhenkirito123’s thread below.
Zhen Wu
@zhenkirito123
Can humanoids perform agile, autonomous, long-horizon parkour—based on what they see in the world?
We present 𝗣𝗲𝗿𝗰𝗲𝗽𝘁𝗶𝘃𝗲 𝗛𝘂𝗺𝗮𝗻𝗼𝗶𝗱 𝗣𝗮𝗿𝗸𝗼𝘂𝗿 (𝗣𝗛𝗣): a framework that chains dynamic human skills using onboard depth perception for long-horizon traversal.
1/6 -
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
Really excited to see the creativity and community engagement around mjlab 😍
Here’s a thread of what people are building 🧵👇 -
Pieter Abbeel reposted
Igor Gilitschenski
@igilitschenski
🚀 Excited to share REPPO, a new on-policy RL agent!
TL;DR: Replace PPO with REPPO for fewer hyperparameter headaches and more robust training.
REPPO, led by @c_voelcker, will be presented at #iclr2026. How does it work? 🧵👇 -
Pieter Abbeel reposted
Brent Yi
@brenthyi
New project! Flow Policy Gradients for Robot Control
tldr; a simple online RL recipe for training and fine-tuning flow policies for robots
co-led w/ @redstone_hong: hongsukchoi.github.io/fpo-control -
Pieter Abbeel reposted
Carlo Sferrazza
@carlo_sferrazza
We introduce a framework that enables robust, long-horizon bi-directional locomotion over complex terrains, by effectively leveraging a single policy with dual-depth camera streams, without the need for LiDAR-based elevation maps.
Check out @Yuanhang__Zhang's tweet for architecture and implementation details, and the website for full, uncut rollouts.
Yuanhang Zhang
@Yuanhang__Zhang
Robust humanoid perceptive locomotion is still underexplored. Especially when different cameras see different terrains, paths get narrow, and payloads disturb balance...
Introduce RPL, tackling this with one unified policy:
• Challenging terrains (slopes, stairs and stepping stones);
• Multiple directions;
• Payloads;
Trained in sim. Validated long-horizon in the real world.
Watch the robot walk it all🦿
Details below👇This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Yuanhang Zhang
@Yuanhang__Zhang
Robust humanoid perceptive locomotion is still underexplored. Especially when different cameras see different terrains, paths get narrow, and payloads disturb balance...
Introduce RPL, tackling this with one unified policy:
• Challenging terrains (slopes, stairs and stepping stones);
• Multiple directions;
• Payloads;
Trained in sim. Validated long-horizon in the real world.
Watch the robot walk it all🦿
Details below👇This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Justin Yu ✈️ CoRL
@uynitsuj
[Accepted to ICRA 2026!] 🚀 Introducing EgoMI: An egocentric manipulation interface that captures synchronized 6-DoF head and hand trajectories from egocentric human demonstrations!
Transfers to IL policies zero-shot w/o visual augmentation or on-embodiment data.
1/n -
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
mjlab v1.0.0 is officially out and considered stable.
Huge thanks to everyone who contributed code, reported issues, and gave feedback. This release wouldn’t have happened without you.
github.com/mujocolab/mjlab -
Pieter Abbeel reposted
Guanya Shi
@GuanyaShi
Very excited to introduce Holosoma, a full-stack fully open-sourced codebase for humanoid sim2real learning:
• Manager-based, easy for extension (we provided a quadruped example)
• Multiple simulators (Isaac Gym, Isaac Sim, MuJoCo Warp)
• Multiple RL algs with minimal rewards: PPO and FastSAC (enables rapid training of locomotion policies in 15 mins with a single 4090)
• Full open-source: retargeting (OmniRetarget), multi-GPU training, sim2sim and sim2real inference
• Support both locomotion and whole-body tracking
• Support both G1 and T1
👉github.com/amazon-far/holosoma
Carlo Sferrazza
@carlo_sferrazza
Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma.
To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference. -
Pieter Abbeel reposted
Carlo Sferrazza
@carlo_sferrazza
All our best results in Holosoma were obtained with FastSAC. We wrote a technical report detailing the key ingredients needed to scale off-policy RL -- both SAC and TD3 -- to massively parallel GPU simulation.
Full-fledged humanoid locomotion in 15 minutes, whole-body tracking much faster than PPO.
younggyo.me/fastsac-humanoid/
Check out Younggyo's thread for more details 👇
Younggyo Seo
@younggyoseo
Tired of waiting hours for humanoids to learn to walk?
Our new technical report shows how to train sim-to-real humanoid locomotion in 15 minutes with FastSAC and FastTD3! The full pipeline is open-source in the newly released Holosoma codebase.
Thread 🧵 -
Pieter Abbeel reposted
Younggyo Seo
@younggyoseo
Tired of waiting hours for humanoids to learn to walk?
Our new technical report shows how to train sim-to-real humanoid locomotion in 15 minutes with FastSAC and FastTD3! The full pipeline is open-source in the newly released Holosoma codebase.
Thread 🧵 -
Pieter Abbeel reposted
Younggyo Seo
@younggyoseo
Excited to release Holosoma, our new full-stack, open-source humanoid learning library from Amazon FAR!
This has been a huge deal for me -- It enables an incredibly fast research cycle. I'm now able to train and deploy a new policy on real hardware in just 20 minutes with an RTX 4090.
You can simply train RL policies, benchmark in Isaacgym, Isaacsim, MuJoCo, and run sim2real deployment with just a few commands in a single codebase!
Carlo Sferrazza
@carlo_sferrazza
Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma.
To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference. -
Pieter Abbeel reposted
Alejandro Escontrela
@alescontrela
Introducing Holosoma (Greek for “whole body”): an easy-to-use and unified control stack that runs across humanoids (G1, T1), quadrupeds (Go2), multiple simulators (MuJoCo, MJLab, IsaacGym/Sim), any terrain, and tasks from velocity control to full-body tracking
Carlo Sferrazza
@carlo_sferrazza
Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma.
To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference. -
Pieter Abbeel
@pabbeel
Open-source: complete codebase covering multiple simulation backends, training, retargeting, and real-world inference. Infra built for humanoid, but also readily modified for quadruped (also included). Lots of infra gems/conveniences we rely on consistently. Hopefully equally helpful for others.
Carlo Sferrazza
@carlo_sferrazza
Sim-to-real learning for humanoid robots is a full-stack problem. Today, Amazon FAR is releasing a full-stack solution: Holosoma.
To accelerate research, we are open-sourcing a complete codebase covering multiple simulation backends, training, retargeting, and real-world inference. -
Pieter Abbeel reposted
Michal Nauman
@mic_nau
Multi-task RL can be highly sample-efficient and when done right, it unlocks LLM-style transfer and fine-tuning.
We’re excited to introduce BRC, a simple recipe for multi-task RL that outperforms SOTA single-task agents while using less compute (!) -
Pieter Abbeel reposted
Sergey Karayev
@sergeykarayev
Ruby/Rails people, if you try it out on superconductor.dev, DM me! We’ll help you set up your dev env if needed.
-
Pieter Abbeel reposted
Zhao-Heng Yin
@zhaohengyin
Introducing Lightning Grasp, a high-performance procedural grasp synthesis algorithm that generates thousands of dexterous grasps in seconds, across diverse hands and challenging objects.
⚡️ 10-100x faster than sota.
Paper: arxiv.org/abs/2511.07418
Code: github.com/zhaohengyin/lightning-grasp -
Pieter Abbeel reposted
Yanjie Ze
@ZeYanjie
Excited to introduce TWIST2, our next-generation humanoid data collection system. TWIST2 is portable (use anywhere, no MoCap), scalable (100+ demos in 15 mins), and holistic (unlock major whole-body human skills).
Fully open-sourced:
yanjieze.com/TWIST2This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Kevin Frans
@kvfrans
What really matters in matrix-whitening optimizers (Shampoo/SOAP/PSGD/Muon)? We ran a careful comparison, dissecting each algorithm.
Interestingly, we find that proper matrix-whitening can be seen as *two* transformations, and not all optimizers implement both.
Blog: kvfrans.com/matrix-whitening/
Paper: arxiv.org/abs/2510.25000 -
Pieter Abbeel reposted
Xueqian Bai
@baicrystal25
How to design soft robot hands for general tasks while incorporating control policies? 💡🦾
Introducing CEM-RM — a hybrid optimization + learned reward model framework that:
- Leverages teleop data to guide design search in sim
- Optimizes efficiently with fewer env interactions -
Pieter Abbeel reposted
Atli Kosson
@AtliKosson
The Maximal Update Parameterization (µP) allows LR transfer from small to large models, saving costly tuning. But why is independent weight decay (IWD) essential for it to work?
We find µP stabilizes early training (like an LR warmup), but IWD takes over in the long term! 🧵 -
Pieter Abbeel reposted
Guanya Shi
@GuanyaShi
An important step towards systematically learning perceptive humanoid policies and humanoid real2sim2real!
Alejandro Escontrela
@alescontrela
Simulation drives robotics progress, but how do we close the reality gap?
Introducing GaussGym: an open-source framework for learning locomotion from pixels with ultra-fast parallelized photorealistic rendering across >4,000 iPhone, GrandTour, ARKit, and Veo scenes!
Thread 🧵 -
Pieter Abbeel reposted
Vikash Kumar ✈️CoRL2026
@Vikashplus
Very cool idea 😎— Generative simulation meets Gaussian splats to bridge the sim2real gap.
Alejandro Escontrela
@alescontrela
Simulation drives robotics progress, but how do we close the reality gap?
Introducing GaussGym: an open-source framework for learning locomotion from pixels with ultra-fast parallelized photorealistic rendering across >4,000 iPhone, GrandTour, ARKit, and Veo scenes!
Thread 🧵 -
Pieter Abbeel reposted
Hongsuk Benjamin Choi
@redstone_hong
The coolest website I have recently seen! 🔥 escontrela.me/gauss_gym/
Love seeing how real-to-sim keeps improving. Super interesting that Veo3 outputs can be “3D-ified” and used for robotics! The paper also ties nicely to VideoMimic, which did real-to-sim and LiDAR-conditioned policies and showed the potential of egocentric rendering.
Alejandro Escontrela
@alescontrela
Simulation drives robotics progress, but how do we close the reality gap?
Introducing GaussGym: an open-source framework for learning locomotion from pixels with ultra-fast parallelized photorealistic rendering across >4,000 iPhone, GrandTour, ARKit, and Veo scenes!
Thread 🧵 -
Pieter Abbeel
@pabbeel
very excited about the upgrade GaussGym provides for environments for training locomotion capabilities
Alejandro Escontrela
@alescontrela
Simulation drives robotics progress, but how do we close the reality gap?
Introducing GaussGym: an open-source framework for learning locomotion from pixels with ultra-fast parallelized photorealistic rendering across >4,000 iPhone, GrandTour, ARKit, and Veo scenes!
Thread 🧵 -
Pieter Abbeel reposted
Alejandro Escontrela
@alescontrela
Simulation drives robotics progress, but how do we close the reality gap?
Introducing GaussGym: an open-source framework for learning locomotion from pixels with ultra-fast parallelized photorealistic rendering across >4,000 iPhone, GrandTour, ARKit, and Veo scenes!
Thread 🧵 -
Pieter Abbeel reposted
Lukas Ziegler
@lukas_m_ziegler
Just a robot doing wallflips! 🤸🏼♀️
Robots start to become more agile than humans.
Most methods for teaching humanoid robots retarget human motion, but they often fail: foot-skating, penetration, and a lack of real human-object interaction.
OmniRetarget solves this by preserving spatial and contact relationships between the robot, terrain, and objects.
Alright, but how? So it minimizes mesh deformation while improving kinematic constraints, producing trajectories that are both feasible and physically consistent.
This makes it possible to scale one demonstration into many variations across robots, terrains, and object setups.
In practice, OmniRetarget generated 8+ hours of high-quality motion, leading to reinforcement learning policies that let a Unitree G1 perform long-horizon parkour and loco-manipulation with only simple rewards and domain randomization.
Great job @pabbeel and team! 🦾
more here: omniretarget.github.ioThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Zhen Wu
@zhenkirito123
I've long wondered if we can make a humanoid robot do a 𝘄𝗮𝗹𝗹𝗳𝗹𝗶𝗽 - and we just made it happen by leveraging 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁 with BeyondMimic tracking!
This came after our original OmniRetarget experiments, with only minor tweaks to RL training: relaxing a termination threshold and removing one reward term.
The policy achieved a 𝟱/𝟱 success rate in our real-world experiments, showing the strength of high-quality, interaction-preserving motion retargeting combined with BeyondMimic’s minimal RL tracking.
Here is the updated arXiv: arxiv.org/abs/2509.26633 (In Sec. V. A)This media couldn't be displayed. See it on the original post.
Zhen Wu
@zhenkirito123
Humanoid motion tracking performance is greatly determined by retargeting quality!
Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL:
- 5 rewards,
- 4 DR terms,
- Proprio. ONLY,
- NO history/curriculum.
Ready for agile, human-like 🤖? (Best with 🎧)
🔗 omniretarget.github.io 🎥
1/9This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
It was a joy bringing Jason’s signature spin-kick to life on the @UnitreeRobotics G1. We trained it in mjlab with the BeyondMimic recipe but had issues on hardware last night (the IMU gyro was saturating). One more sim-tuning pass and we nailed it today. With @qiayuanliao and @pabbeel
Jason Peng
@xbpeng4
Implementing motion imitation methods involves lots of nuisances. Not many codebases get all the details right. So, we're excited to release MimicKit!
github.com/xbpeng/MimicKit
A framework with high quality implementations of our methods: DeepMimic, AMP, ASE, ADD, and more to come! -
Pieter Abbeel reposted
Reflection
@reflection_ai
Today we're sharing the next phase of Reflection.
We're building frontier open intelligence accessible to all.
We've assembled an extraordinary AI team, built a frontier LLM training stack, and raised $2 billion.
Why Open Intelligence Matters
Technological and scientific progress is driven by values of openness and collaboration.
The internet, Linux, and the protocols and standards that underpin modern computing are all open. This isn't a coincidence. Open software is what gets forked, customized, and embedded into systems worldwide. It's what universities teach, what startups build on, what enterprises deploy.
Open science enables others to learn from the results, be inspired by them, interrogate them, and build upon them in order to push the frontier of human knowledge and scientific advancement. AI got to where it is today through scaling ideas (e.g. self-attention, next token prediction, reinforcement learning) that were shared and published openly.
Now AI is becoming the technology layer that everything else runs on top of. The systems that accelerate scientific research, enhance education, optimize energy usage, supercharge medical diagnoses, and run supply chains will all be built on AI infrastructure.
But the frontier is currently concentrated in closed labs. If this continues, a handful of entities will control the capital, compute, and talent required to build AI, creating a runaway dynamic that locks everyone else out. There's a narrow window to change this trajectory. We need to build open models so capable that they become the obvious choice for users and developers worldwide, ensuring the foundation of intelligence remains open and accessible rather than controlled by a few.
What We've Built
Over the last year, we've been preparing for this mission.
We’ve assembled a team who have pioneered breakthroughs including PaLM, Gemini, AlphaGo, AlphaCode, AlphaProof, and contributed to ChatGPT and Character AI, among many others.
We built something once thought possible only inside the world’s top labs: a large-scale LLM and reinforcement learning platform capable of training massive Mixture-of-Experts (MoEs) models at frontier scale. We saw the effectiveness of our approach first-hand when we applied it to the critical domain of autonomous coding. With this milestone unlocked, we're now bringing these methods to general agentic reasoning.
We've raised significant capital and identified a scalable commercial model that aligns with our open intelligence strategy, ensuring we can continue building and releasing frontier models sustainably. We are now scaling up to build open models that bring together large-scale pretraining and advanced reinforcement learning from the ground up.
Safety and Responsibility
Open intelligence also changes how we think about safety. It enables the broader community to participate in safety research and discourse, rather than leaving critical decisions to a few closed labs. Transparency allows independent researchers to identify risks, develop mitigations, and hold systems accountable in ways that closed development cannot.
But openness also requires confronting the challenges of capable models being widely accessible. We're investing in evaluations to assess capabilities and risks before release, security research to protect against misuse, and responsible deployment standards. We believe the answer to AI safety is not “security through obscurity” but rigorous science conducted in the open, where the global research community can contribute to solutions rather than a handful of companies making decisions behind closed doors.
Join Us
There is a window of opportunity today to build frontier open intelligence, but it is closing and this may be the last. If this mission resonates, join us. -
Pieter Abbeel
@pabbeel
never miss a @xbpeng4 release
Jason Peng
@xbpeng4
Implementing motion imitation methods involves lots of nuisances. Not many codebases get all the details right. So, we're excited to release MimicKit!
github.com/xbpeng/MimicKit
A framework with high quality implementations of our methods: DeepMimic, AMP, ASE, ADD, and more to come! -
Pieter Abbeel
@pabbeel
ResMimic: learns a whole-body loco-manipulation policy on top of general motion tracking a policy
Key ideas:
(i) pre-train general motion tracking
(ii) post-train task-specific residual policy with:
(a) object tracking reward
(b) contact reward
(c) virtual object force curriculum
Siheng Zhao✈️CoRL
@SihengZhao
ResMimic: a two-stage residual framework that unleashes the power of pre-trained general motion tracking policy.
Enable expressive whole-body loco-manipulation with payloads up to 5.5kg without task-specific design, generalize across poses, and exhibit reactive behavior. -
Pieter Abbeel reposted
Thomas Wolf
@Thom_Wolf
Wow! Super impressive work by the new Amazon FAR team (from Covariant acquisition).
Mapping long sequences of human motion (>30 sec) on robots with a differing shapes or interating with objects (box, table, etc) of different size. Enabling easier in-simulation data-augmentation and zero-shoot transfer.
Super impressive and huge help to reduce the need for human teleop data (which is very complex to gather for humanoids)
Dataset trajectories on Hugging Face (search OmniRetarget), full code framework to come soon
Project page has some pretty three.js interactive demos -
Pieter Abbeel
@pabbeel
Welcome Nikita, Iurii, Eldar, Laurynas, super-excited to be working together going forward!
Cotracker
VGGT
Deepcut
Deepercut
Posetrack
Text-to-4d
Clevrtex
...........
💥💥💥💥💥💥💥💥💥
Nikita Karaev
@n_karaev
🚨Big news - our team is joining the Frontier AI and Robotics (FAR) lab at @amazon to keep building the future of robotics. Excited to be joining such an inspiring team and to work with @peterxichen, @pabbeel, @rocky_duan, @akanazawa and many others at FAR.
-
Pieter Abbeel reposted
Qianzhong Chen ✈️ CoRL2026
@QianzhongChen
🚀 Introducing SARM: Stage-Aware Reward Modeling for Long-Horizon Robot Manipulation
Robots struggle with tasks like folding a crumpled T-shirt—long, contact-rich, and hard to label. We propose a scalable reward modeling framework to fix that. 1/n -
Pieter Abbeel reposted
Sherry Yang
@sherryyangML
We have developed a suite of tasks for evaluating policies in a world model which we call WorldGym. We use WorldGym to compare a few different policies (OpenVLA, Octo, RT-1-X).
Sherry Yang
@sherryyangML
Evaluating policies on a real robot can be painful. Can we use a world model to get a rough estimate of how good a policy is?
Checkout "Evaluating Robot Policies in a World Model".
Paper: arxiv.org/abs/2506.00613
Demo: world-model-eval.github.io
Code: github.com/world-model-eval/world-model-eval -
Pieter Abbeel reposted
Carlo Sferrazza
@carlo_sferrazza
Very excited to share OmniRetarget! We advance humanoid motion tracking by fixing retargeting at its source -- generating high-quality, interaction-preserving trajectories that are much easier for RL to track.
Zhen Wu
@zhenkirito123
Humanoid motion tracking performance is greatly determined by retargeting quality!
Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL:
- 5 rewards,
- 4 DR terms,
- Proprio. ONLY,
- NO history/curriculum.
Ready for agile, human-like 🤖? (Best with 🎧)
🔗 omniretarget.github.io 🎥
1/9This media couldn't be displayed. See it on the original post. -
Pieter Abbeel
@pabbeel
Great initiative, very exciting to see LightSpeed provide funding for the fundamental AI and Systems research at Berkeley that has been originating so many (open source!) research breakthroughs and start-ups
There is a lot of talk about paying it forward, and here is Lightspeed actually paying it forward! ♥️♥️♥️
PS: $ going into this kind of lab are almost surely the highest leverage in terms of
"open source research frontier progress" / "$"
not to mention grow talent pool
James Alcorn
@JamesAlcorn94
more funding for research - no strings attached.
cc @istoica05, @profjoeyg, @pabbeel, @f_fatemi21, @ravi_lsvp
lsvp.com/stories/lightspeed-sky-lab-supporting-uc-berkele… -
Pieter Abbeel reposted
Peter Chen
@peterxichen
New work from Amazon FAR: with high-quality data, humanoids can perform a wide range of agile whole-body motion and loco-manipulation tasks with a very minimal RL & sim2real recipe. Check out website omniretarget.github.io for the range of tasks that can be performed by the same method.
Zhen Wu
@zhenkirito123
Humanoid motion tracking performance is greatly determined by retargeting quality!
Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL:
- 5 rewards,
- 4 DR terms,
- Proprio. ONLY,
- NO history/curriculum.
Ready for agile, human-like 🤖? (Best with 🎧)
🔗 omniretarget.github.io 🎥
1/9This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Guanya Shi
@GuanyaShi
High-quality motion reference data is key for humanoid skill learning 🤖🕺💃
A natural idea is to leverage human motions and “translate” them to humanoid motions, a process known as retargeting. For interaction-rich tasks such as scene interaction and loco-manipulation, retargeting is challenging: it must ensure motion consistency, smoothness, kinematic feasibility (no artifacts like penetration or foot skating), and scalability (one framework can handle thousands of motions).
Excited to release OmniRetarget — a scalable retargeting method with a 4-hour high-quality humanoid motion dataset for interaction-rich tasks. OmniRetarget takes an interaction-preserving perspective: we optimize Laplacian deformation between source and target interaction meshes while enforcing kinematic constraints, producing consistent, smooth, and feasible trajectories at scale. Even better, OmniRetarget can efficiently augment motions by varying terrains, objects, and initial poses.
This high-quality interaction-preserving retargeting enables a minimal RL setup to execute long-horizon (up to 30s) agile, interaction-rich skills. All tasks in the video share just 5 rewards, 4 domain randomization terms, and rely only on proprioception.
More details: omniretarget.github.io
Zhen Wu
@zhenkirito123
Humanoid motion tracking performance is greatly determined by retargeting quality!
Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL:
- 5 rewards,
- 4 DR terms,
- Proprio. ONLY,
- NO history/curriculum.
Ready for agile, human-like 🤖? (Best with 🎧)
🔗 omniretarget.github.io 🎥
1/9This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Younggyo Seo
@younggyoseo
I've recently started working on humanoids and this project set the bar super high for me 😂 Very impressive demo, and our team is working on many interesting projects, exciting time!
Zhen Wu
@zhenkirito123
Humanoid motion tracking performance is greatly determined by retargeting quality!
Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL:
- 5 rewards,
- 4 DR terms,
- Proprio. ONLY,
- NO history/curriculum.
Ready for agile, human-like 🤖? (Best with 🎧)
🔗 omniretarget.github.io 🎥
1/9This media couldn't be displayed. See it on the original post. -
Pieter Abbeel
@pabbeel
Very excited to start sharing some of the work we have been doing at Amazon FAR. In this work we present OmniRetarget, which can generate high-quality interaction-preserving data from human motions for learning complex humanoid skills.
High-quality re-targeting really helps the reinforcement learning.
Why? The control policy optimization landscape is much nicer for (near)feasible trajectories than for trajectories with artifacts like (e.g.) foot skating or ground penetration.
Zhen Wu
@zhenkirito123
Humanoid motion tracking performance is greatly determined by retargeting quality!
Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL:
- 5 rewards,
- 4 DR terms,
- Proprio. ONLY,
- NO history/curriculum.
Ready for agile, human-like 🤖? (Best with 🎧)
🔗 omniretarget.github.io 🎥
1/9This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Christopher Manning
@chrmanning
Thrilled to see former @StanfordAILab students @sunfanyun and my own student @sharonal_lee today launching @moonlake, a new AI lab that lets anyone code interactive worlds in minutes.
@aixventureshq led their $28M seed along @thresholdvc, @NVIDIA Ventures, and leading angels including @stevechen and @JeffDean.
@RashiShrivast18 at @Forbes covers the news: www.forbes.com/sites/rashishrivastava/2025/10/01/gaming-w… -
Pieter Abbeel reposted
Zhen Wu
@zhenkirito123
Humanoid motion tracking performance is greatly determined by retargeting quality!
Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL:
- 5 rewards,
- 4 DR terms,
- Proprio. ONLY,
- NO history/curriculum.
Ready for agile, human-like 🤖? (Best with 🎧)
🔗 omniretarget.github.io 🎥
1/9This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Lars Ankile
@larsankile
How can we enable finetuning of humanoid manipulation policies, directly in the real world?
In our new paper, Residual Off-Policy RL for Finetuning BC Policies, we demonstrate real-world RL on a bimanual humanoid with 5-fingered hands (29 DoF) and improve pre-trained policies with ~15-75 minutes of robot interaction.
By learning residual corrections on frozen BC policies using sample-efficient off-policy RL, we achieve significant improvements in sample efficiency, enabling policy finetuning directly on the hardware — to our knowledge, one of the first examples of this on a humanoid with bimanual dexterous hands.
(If you know of other examples, let me know!)This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
I'm super excited to announce mjlab today!
mjlab = Isaac Lab's APIs + best-in-class MuJoCo physics + massively parallel GPU acceleration
Built directly on MuJoCo Warp with the abstractions you love. -
Pieter Abbeel reposted
Ritvik Singh
@ritvik_singh9
Our latest work performs sim2real dexterous grasping using end-to-end depth RL.
This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Carlo Sferrazza
@carlo_sferrazza
Excited to share that I'll be joining @UTAustin in Fall 2026 as an Assistant Professor with @utmechengr @texas_robotics!
I'm looking for PhD students interested in humanoids, dexterous manipulation, tactile sensing, and robot learning in general -- consider applying this cycle! -
Pieter Abbeel reposted
Reads with Ravi
@readswithravi
Jensen Huang on why he don’t wear a watch:
-
Pieter Abbeel reposted
Preston Fu
@preston_fu
If we have tons of compute to spend to train value functions, how can we be sure we're spending it optimally? In our new paper, we analyze the interplay of model size, UTD, and batch size for training value functions achieving optimal performance.
arxiv.org/abs/2508.14881 -
Pieter Abbeel reposted
Rocky Duan
@rocky_duan
We're hiring interns (and full-times) all year long! Please email me if interested.
-
Pieter Abbeel reposted
Robert Nishihara
@robertnishihara
Congratulations to my brilliant co-founder Philipp Moritz (@pcmoritz) and the legendary John Schulman, Sergey Levine, Pieter Abbeel, and Michael Jordan on their Test-of-Time Honorable Mention at ICML 2025 today!
For creating TRPO. This was done during the previous wave of excitement around reinforcement learning (Atari, Mujoco, AlphaGo, etc). Now RL is back thanks to LLMs and reasoning models and the work is more impactful than ever before! The field has to a large degree standardized on variants of TRPO.
@pcmoritz @johnschulman2, @svlevine, @pabbeel -
Pieter Abbeel reposted
Raquel Urtasun
@RaquelUrtasun
Today we're unveiling something truly extraordinary—one of the coolest and most transformative technologies: Mixed Reality Testing (MRT). Since launching @Waabi_ai World four years ago, we've been building towards this pivotal moment - bringing our virtual testing capabilities into the physical world to fundamentally transform what's possible in autonomous vehicle development.
For over 100 years, closed-course testing has been the staple of vehicle safety testing. But creating scenarios on a physical test track resembles elaborate movie-stunts—it’s difficult to reproduce tests consistently, those tests do not capture the diversity of the real-world, and the range of safety-critical scenarios we can test is very limited because of the risk of damaging equipment or, worse, endangering lives.
MRT changes everything! In the same way that augmented reality goggles blend the physical world with a virtual world, MRT enables the Waabi Driver to drive autonomously down a physical test track while simultaneously experiencing numerous intelligent, simulated actors that coexist in this hybrid reality and react to each other and to the physical world in naturalistic ways. All this is possible by leveraging Onboard Waabi World, a version of Waabi's neural simulator that runs in a few milliseconds on the onboard compute. As Onboard Waabi World generates new scenarios, the real physical sensor readings are modified instantaneously so the Waabi Driver can react to the blend of real and virtual elements while driving in the physical world. This fusion creates a first-of-its-kind reality that unlocks unlimited testing possibilities previously impossible to achieve safely or practically.
Traffic jams can materialize instantly, motorcycles can weave between lanes, crowds of pedestrians can jaywalk unpredictably, children can dart into the street from behind parked cars, animals can wander across the road, debris can appear in lanes creating hazardous obstacles. The possibilities are endless.
This breakthrough eliminates all traditional constraints of closed-course testing. We can do exponentially more tests than ever before and in a continuous fashion, with extraordinary levels of diversity and realism. Most importantly, we can push the boundaries of safety testing without real-world consequences.
MRT represents the culmination of years of work toward safe, scalable autonomous deployment. It's been central to our testing approach for over two years and fundamental to achieving feature-complete autonomous driving capabilities earlier this year.
But this is just the beginning. MRT’s impact extends beyond self-driving trucks – we’re laying the foundation for a future where any physical AI system can be tested and validated safely - from robotaxis to humanoid robots, manufacturing automation to healthcare robotics.
Learn about Mixed Reality Testing: waabi.ai/mixed-reality-testing-pushes-the-boundaries-of-a…This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Shane Gu
@shaneguML
Congratulations Misha! Something many engineers in enterprises wanted for a long time. A great example of unified research-modeling-product-business strategy :)
-
Pieter Abbeel reposted
Misha Laskin
@MishaLaskin
Engineers spend 70% of their time understanding code, not writing it.
That’s why we built Asimov at @reflection_ai.
The best-in-class code research agent, built for teams and organizations.This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Haoran Geng
@HaoranGeng2
🤖 What if a humanoid robot could make a hamburger from raw ingredients—all the way to your plate?
🔥 Excited to announce ViTacFormer: our new pipeline for next-level dexterous manipulation with active vision + high-resolution touch.
🎯 For the first time ever, we demonstrate ~2.5 minutes of continuous, autonomous control—combining active vision, high-res touch, and high-DoF robot hands SharpaWave — to complete complex, real-world tasks.
Code is fully released; check out our:
Homepage: roboverseorg.github.io/ViTacFormerPage/
Paper link: arxiv.org/abs/2506.15953
Github: github.com/RoboVerseOrg/ViTacFormerThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
pfung
@philfung
if you want to try training a robot to dance or pick up stuff, check out these Colab notebooks released by Google & @kevin_zakka this week.
Train Unitree dog to spin and handstand: colab.research.google.com/github/google-deepmind/mujoco_p…
Train Franka robot arm to pick up stuff:
colab.research.google.com/github/google-deepmind/mujoco_p… -
Pieter Abbeel reposted
Berkeley AI Research
@berkeley_ai
Congratulations to BAIR researchers @kevin_zakka @qiayuanliao @arthurallshire @carlo_sferrazza @KoushilSreenath @pabbeel and Google collaborators for winning the Outstanding Demo Paper Award at RSS 2025!
playground.mujoco.org
Kevin Zakka
@kevin_zakka
We’re super thrilled to have received the Outstanding Demo Paper Award for MuJoCo Playground at RSS 2025!
Huge thanks to everyone who came by our booth and participated, asked questions, and made the demo so much fun!
@carlo_sferrazza @qiayuanliao @arthurallshire -
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
Demo starting in 10 minutes, come witness the magic of open-source sim2real!
-
Pieter Abbeel reposted
Carlo Sferrazza
@carlo_sferrazza
Sim2real is getting so mature that with great hardware (thanks @clemens_chr @katzschmann), you can get things running in just days 🤖
We are also demoing @orcahand trained with MuJoCo Playground at #rss2025. Come by Associates Park at 12:30pm-2pm and 6:30pm-8pm!
@RoboticsSciSysThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Zhao-Heng Yin
@zhaohengyin
Just open-sourced Geometric Retargeting (GeoRT) — the kinematic retargeting module behind DexterityGen.
Includes tools for importing custom hands.
Give it a try: github.com/facebookresearch/GeoRT
A software by @berkeley_ai and @AIatMeta. More coming soon. -
Pieter Abbeel reposted
Kevin Frans
@kvfrans
Very excited for this one. We took a cautiously experimental view on NN optimizers, aiming to find something that just works.
SPlus matches Adam within ~44% of steps on a range of objectives. Please try it out in your setting, or read below for how it works. -
Pieter Abbeel reposted
Ademi Adeniji
@AdemiAdeniji
Everyday human data is robotics’ answer to internet-scale tokens.
But how can robots learn to feel—just from videos?📹
Introducing FeelTheForce (FTF): force-sensitive manipulation policies learned from natural human interactions🖐️🤖
👉 feel-the-force-ftf.github.io
1/n -
Pieter Abbeel reposted
Haoran Geng
@HaoranGeng2
🚀Check out our new work, FastTD3, a reinforcement learning algorithm that is simple, efficient, and highly capable.
It achieves truly remarkable performance across challenging RL tasks.
Younggyo Seo
@younggyoseo
Excited to present FastTD3: a simple, fast, and capable off-policy RL algorithm for humanoid control -- with an open-source code to run your own humanoid RL experiments in no time!
Thread below 🧵This media couldn't be displayed. See it on the original post. -
Pieter Abbeel
@pabbeel
FastTD3: "Minimum innovation, maximum results"
Not the paper we had planned to write, but one of the works I am most proud of. We wanted to make sure our baseline (TD3) was a very solid baseline, so we added a few things that are already known to help in RL (large, distributional critic, massive parallel, large batch size) to original TD3 and we ended up with a baseline we couldn't beat anymore 😂😅
Open-source, kept maximally simple, highly recommend to try it out for any humanoid (or similar) RL experiments!
🥳🚀🥳🚀🥳🚀🥳🚀
The starting quote is what @ilyasut used to say at @OpenAI in the early years when I was there, great wisdom
Younggyo Seo
@younggyoseo
Excited to present FastTD3: a simple, fast, and capable off-policy RL algorithm for humanoid control -- with an open-source code to run your own humanoid RL experiments in no time!
Thread below 🧵This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Younggyo Seo
@younggyoseo
Excited to present FastTD3: a simple, fast, and capable off-policy RL algorithm for humanoid control -- with an open-source code to run your own humanoid RL experiments in no time!
Thread below 🧵This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Carlo Sferrazza
@carlo_sferrazza
Off-policy learning transfers from sim to real-world humanoids!
Off-policy methods have pushed RL sample efficiency, but robotics still leans on parallel on-policy RL (PPO) for wall-time gains. Introducing FastTD3, a simple off-policy algorithm that gets the best of both worlds!
Younggyo Seo
@younggyoseo
Excited to present FastTD3: a simple, fast, and capable off-policy RL algorithm for humanoid control -- with an open-source code to run your own humanoid RL experiments in no time!
Thread below 🧵This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Kevin Frans
@kvfrans
Over the past year, I've been compiling some "alchemist's notes" on deep learning. Right now it covers basic optimization, architectures, and generative models.
Focus is on learnability -- each page has nice graphics and an end-to-end implementation.
notes.kvfrans.com -
Pieter Abbeel reposted
Vincent Liu
@vincentjliu
The future of robotics isn't in the lab – it's in your hands. Can we teach robots to act in the real world without a single robot demonstration?
Introducing EgoZero. Train real-world robot policies from human-first egocentric data. No robots. No teleop. Just Aria glasses and humans.
w/ @ademiadeniji @haotianz_david @pabbeel @lerrelpinto
1/n 🧵👇This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Ademi Adeniji
@AdemiAdeniji
Closed-loop robot policies directly from human interactions. No teleop, no robot data co-training, no RL, and no sim. Just Aria smart glasses.
Everyday human data is passively scalable and a massively underutilized resource in robotics...More to come here in the coming weeks.
Vincent Liu
@vincentjliu
The future of robotics isn't in the lab – it's in your hands. Can we teach robots to act in the real world without a single robot demonstration?
Introducing EgoZero. Train real-world robot policies from human-first egocentric data. No robots. No teleop. Just Aria glasses and humans.
w/ @ademiadeniji @haotianz_david @pabbeel @lerrelpinto
1/n 🧵👇This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Rocky Duan
@rocky_duan
Our robotics team will be at ICRA next week in Atlanta! Having started a new research team at Amazon building robot foundation models, we're hiring across all levels, full-time or intern, and across both SW and Research roles. Ping me at drockyd@amazon.com and let's have a chat!
-
Pieter Abbeel reposted
Arthur Allshire
@arthurallshire
our new system trains humanoid robots using data from cell phone videos, enabling skills such as climbing stairs and sitting on chairs in a single policy
(w/ @redstone_hong @junyi42 @davidrmcall)This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Berkeley Computing, Data Science, and Society
@BerkeleyCDSS
Two members of the @UCBerkeley computer science community—Ion Stoica (@istoica05) and John Schulman (@johnschulman2)—are recognized with UC Berkeley Achievement Awards! 🎉 Presented by UC Berkeley Foundation and Cal Alumni Association. @Berkeley_EECS
cdss.berkeley.edu/news/ion-stoica-and-john-schulman-recog… -
Pieter Abbeel
@pabbeel
I'm thrilled to share our first release as the AGI SF Lab. Meet Nova Act -- the most effortless way to build agents that can reliably use browsers, giving agents access to much of our digital world.
It brings us closer to building universal agents in both digital and physical world.
See what Nova Act can do:This media couldn't be displayed. See it on the original post.
Pieter Abbeel
@pabbeel
When interleaved with Python code, Nova Act lets you MapReduce over the entire internet.
You can run agents in parallel using thread pools, extract what’s on the page into Pydantic classes, and even have direct Playwright access.Interested?
Sign up on our product page to get your API key!
nova.amazon.com/act
Pieter Abbeel
@pabbeel
And, last but not least, this is just the beginning, AGI SF lab is actively recruiting!
www.amazon.science/blog/amazon-opens-new-ai-lab-in-san-fr… -
Pieter Abbeel reposted
Amazon Science
@AmazonScience
Meet Amazon Nova Act — an effortless way to build AI agents that can reliably use browsers 🧑💻
With our new model, compose robust steps into complex workflows; handle everything from bookings to QA testing. Getting started takes just 3 lines of code.
See what Nova Act can do 🧵👇This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Berkeley AI Research
@berkeley_ai
BAIR alum and @perplexity_ai CEO @AravSrinivas seeks to cater "... to curiosity at the scale of humanity..."
engineering.berkeley.edu/news/2025/03/berkeley-alum-wants… -
Pieter Abbeel
@pabbeel
My knee-jerk reaction was that sim-2-real wouldn't work watching the learned policy in sim (on left), but actually it does!
Kevin Zakka
@kevin_zakka
Booster recovery controller from last night. Sim design, training and deployment on hardware took < 1 day. With @qiayuanliao
-
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
Booster recovery controller from last night. Sim design, training and deployment on hardware took < 1 day. With @qiayuanliao
-
Pieter Abbeel reposted
Fangchen Liu
@fangchenliu_
1/N Most Vision-Language-Action models need tons of data for finetuning, and still fail for new objects and instructions. Introducing OTTER, a lightweight, easy-to-train model that uses text-aware visual features to nail unseen tasks out of the box! Here's how it works 👇
-
Pieter Abbeel
@pabbeel
Founders who were PhD or post-doc in my lab at Berkeley, **largely funded by NSF / DoD grants**, start-up, market cap (collected by OpenAI Deep Research)
-
Pieter Abbeel reposted
Misha Laskin
@MishaLaskin
Today I’m launching @reflection_ai with my friend and co-founder @real_ioannis.
Our team pioneered major advances in RL and LLMs, including AlphaGo and Gemini.
At Reflection, we're building superintelligent autonomous systems. Starting with autonomous coding. -
Pieter Abbeel
@pabbeel
Basics of Deep RL tutorial I am still very happy with, as good a day as any to re-post :)
www.youtube.com/playlist?list=PLwRJQ4m4UJjNymuBM9RdmB3Z9N… -
Pieter Abbeel
@pabbeel
visual intelligence graduating from your phone to your home!
-
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
We've just added the @boosterobotics T1 Humanoid to MuJoCo Playground, with code to train a joystick controller as well as a deployable ONNX file. A super fun weekend with @carlo_sferrazza and @qiayuanliao :)
github.com/google-deepmind/mujoco_playground/tree/main/mu…This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Oleg Rybkin
@_oleh
Does off-policy value-based RL scale? In LLMs, larger scale predictably improves performance. Value-based RL learns from arbitrary data and is sample-efficient, but folk wisdom says it doesn't scale
🧵⬇️We show predictability for scaling value-based RL! arxiv.org/abs/2502.04327 -
Pieter Abbeel reposted
Zhao-Heng Yin
@zhaohengyin
We introduce Dexterity Gen (DexGen), a foundation controller that enables unprecedented dexterous manipulation capabilities. For the first time, it allows human teleoperation of tasks such as using a pen, screwdriver, and syringe. Developed by @berkeley_AI and @MetaAI. A Thread.
-
Pieter Abbeel reposted
Anthony Goldbloom
@antgoldbloom
Tomorrow between 11am and 2pm, @chrmanning, the world's most cited NLP researcher, is running a session on Deepseek and it's implications. Here's a link if you want to join: lu.ma/vvkddaz9
-
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
The ultimate test of any physics simulator is its ability to deliver real-world results.
With MuJoCo Playground, we’ve combined the very best: MuJoCo’s rich and thriving ecosystem, massively parallel GPU-accelerated simulation, and real-world results across a diverse range of robot platforms: quadrupeds, humanoids, dexterous hands, and arms.
Best of all? You can get started today with a single command: pip install playground
playground.mujoco.orgThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Carlo Sferrazza
@carlo_sferrazza
Big news for open-source robot learning! We are very excited to announce MuJoCo Playground.
The Playground is a reproducible sim-to-real pipeline that leverages MuJoCo ecosystem and GPU acceleration to learn robot locomotion and manipulation in minutes.
playground.mujoco.org
Kevin Zakka
@kevin_zakka
The ultimate test of any physics simulator is its ability to deliver real-world results.
With MuJoCo Playground, we’ve combined the very best: MuJoCo’s rich and thriving ecosystem, massively parallel GPU-accelerated simulation, and real-world results across a diverse range of robot platforms: quadrupeds, humanoids, dexterous hands, and arms.
Best of all? You can get started today with a single command: pip install playground
playground.mujoco.orgThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Oier Mees
@oier_mees
Can generalist robot 🤖 policies understand heterogeneous multimodal sensors?
Introducing FuSe, a novel finetuning recipe allowing VLAs to reason across 👁️ vision, 🤏touch, 🔊 sound & 🗣️ language, enabling queries "pick the object that feels like🍍 but is brown" (hint: pinecone) -
Pieter Abbeel reposted
Younggyo Seo
@younggyoseo
Introducing CoordTok, a scalable video tokenizer that can encode a 128-frame video into only 1k tokens. CoordTok learns a mapping from (x, y, t) coordinates to the corresponding patches of input videos. 🧵[1/6]
project page: huiwon-jang.github.io/coordtok/ -
Pieter Abbeel reposted
Carlo Sferrazza
@carlo_sferrazza
🚨 New reinforcement learning algorithms 🚨
Excited to announce MaxInfoRL, a class of model-free RL algorithms that solves complex continuous control tasks (including vision-based!) by steering exploration towards informative transitions.
Details in the thread 👇 -
Pieter Abbeel reposted
Miles Brundage
@Miles_Brundage
I recently had the honor of giving a guest lecture in Berkeley's Intro to AI course, CS 188. I gave a broad overview of AI policy (what it is + some reasonably-agreed-upon foundations) as well as sharing some spicier takes. You can find the video here: www.youtube.com/watch?v=JavGesatkeA
-
Pieter Abbeel reposted
David Luan
@jluan
By putting together a super talent-dense team focused on a small number of targeted research bets, we’re able to maximize the amount of compute per capita -- aka do more wild experiments faster.
-
Pieter Abbeel reposted
David Luan
@jluan
!! @pabbeel and I are building a new AI research lab in SF for Amazon! We’re focused on the remaining major problems to build generally intelligent agents and are looking for a few dozen intrinsically motivated people to join our team and work with the Adept folks here. DM me!
Pieter Abbeel
@pabbeel
Super-excited about what's ahead. Want to move the AI research frontier, join us!
www.amazon.science/blog/amazon-opens-new-ai-lab-in-san-fr…
AGI-SFLab-Jobs@amazon.com -
Pieter Abbeel
@pabbeel
Super-excited about what's ahead. Want to move the AI research frontier, join us!
www.amazon.science/blog/amazon-opens-new-ai-lab-in-san-fr…
AGI-SFLab-Jobs@amazon.com -
Pieter Abbeel reposted
Amy Lu
@amyxlu
1/🧬 Excited to share PLAID, our new approach for co-generating sequence and all-atom protein structures by sampling from the latent space of ESMFold. This requires only sequences during training, which unlocks more data and annotations:
www.biorxiv.org/content/10.1101/2024.12.02.626353v1
🧵 -
Pieter Abbeel
@pabbeel
really nicely done collection of illustrations / videos and story!
IEEE Spectrum
@IEEESpectrum
We've all dreamed that a robot will soon cook and clean for us. So where's our robot butler? spectrum.ieee.org/ai-robots?share_id=8436952
-
Pieter Abbeel reposted
Anthony Goldbloom
@antgoldbloom
Looking at job posts over the last year, it looks like Amazon is making the broadest AI investment. 228 teams across the company posted jobs with GenAI projects over the past year, considerably more than any other company. blog.sumble.com/companies-with-the-broadest-genai-focus/
Most interest is the breadth of teams and use cases. They made posts from the core research teams, Alexa, customer service, seller experience, fulfillment, and catalog selection teams (full list: sumble.com/l/JrsA4zNSp4).
Some examples:
- Creative X Team: generating high-quality text, images and video for advertisers (sumble.com/l/EHF2RZQRYD)
- Geospatial Science Team: for global address parsing and validation (sumble.com/l/h7GWV4t9JU)
- Selection and Catalog Systems Team: improve the completeness and correctness of product data for Amazon shoppers (sumble.com/l/uaddsqaD4v) -
Pieter Abbeel reposted
Anthony Goldbloom
@antgoldbloom
The overlooked GenAI use case: cleaning, processing, and analyzing data.
blog.sumble.com/the-overlooked-genai-use-case/
Job post data tell us what companies plan to do with GenAI. The most common use case is data analytics projects. Examples:
- AstraZeneca: using LLMs on freeform documents to structure results from their Extractables & Leachables testing (sumble.com/l/FgehjrgnvN)
- Trafigura: The Document AI team is using LLMs to extract data from a corpus of commodity trading documents to generate credit reports
(sumble.com/l/6bY7mhyAHd)
The startup ecosystem is overlooking this use case, instead focusing on other areas such as customer support, sales & marketing and code gen. -
Pieter Abbeel
@pabbeel
Great ICLR 2024 talk on "The Emerging Science of Benchmarks" by Moritz Hardt (which I somehow didn't see, till now):
iclr.cc/virtual/2024/invited-talk/21799 -
Pieter Abbeel reposted
Joey Gonzalez
@profjoeyg
My former students @genmoai just crushed OpenAI with their new video generation model AND they are open-sourcing it!
As an advisor to @genmoai I have to ask WHY ARE YOU GIVING THIS MODEL AWAY?! 😂
Genmo
@genmoai
Introducing Mochi 1 preview. A new SOTA in open-source video generation. Apache 2.0.
magnet:?xt=urn:btih:441da1af7a16bcaa4f556964f8028d7113d21cbb&dn=weights&tr=udp://tracker.opentrackr.org:1337/announceThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Ideogram
@ideogram_ai
Today, we’re introducing Ideogram Canvas, an infinite creative board for organizing, generating, editing, and combining images.
Bring your face or brand visuals to Ideogram Canvas and use industry-leading Magic Fill and Extend to blend them with creative, AI-generated content. -
Pieter Abbeel reposted
-
Pieter Abbeel reposted
Genmo
@genmoai
Introducing Mochi 1 preview. A new SOTA in open-source video generation. Apache 2.0.
magnet:?xt=urn:btih:441da1af7a16bcaa4f556964f8028d7113d21cbb&dn=weights&tr=udp://tracker.opentrackr.org:1337/announceThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Gradio
@Gradio
Mochi 1
Dramatically closes the gap between closed and open video generation models. ✅
Apache 2.0 license 🤯
High-fidelity videos
Strong prompt adherence
Model available on 🤗 Hub -
Pieter Abbeel reposted
Kevin Frans
@kvfrans
*Shortcut models* are a plug-and-play replacement for diffusion models that can generate in a single step (or more). This speeds up inference by up to 128x.
Shortcut models are trained end-to-end, and do not require a separate distillation phase or learning schedules. -
Pieter Abbeel reposted
Hao Liu
@haoliuhl
Fixed token allocation per frame in multimodal models is inefficient. Introduce ElasticTok: adaptive encoding that uses variable tokens based on complexity. It conditions on prior frames, uses scalable masking technique.
Project: largeworldmodel.github.io/elastictok/ -
Pieter Abbeel reposted
Wilson Yan
@wilson1yan
We are excited to announce ElasticTok, a simple yet scalable visual tokenizer that can adaptively encode image and video to variable-length sequences! Our model enables more efficient token usage over different images and long dynamic videos.
-
Pieter Abbeel
@pabbeel
One of my personal favorite conversations of all time, with (now) Nobel Prize winner @geoffreyhinton
youtu.be/2EDP4v-9TUA
Don't miss the part where he was a carpenter in London for a year but concluded he wasn't as good a carpenter as he should be to continue down that path.. -
Pieter Abbeel reposted
Yoshua Bengio
@Yoshua_Bengio
@HopfieldJohn and @geoffreyhinton, along with collaborators, have created a beautiful and insightful bridge between physics and AI. They invented neural networks that were not only inspired by the brain, but also by central notions in physics such as energy, temperature, system dynamics, energy barriers, the role of randomness and noise, connecting the local properties, e.g., of atoms or neurons, to global ones like entropy and attractors. And they went beyond the physics to show how these ideas could give rise to memory, learning and generative models; concepts which are still at the forefront of modern AI research.
Their ideas inspired me so profoundly that I decided to choose learning in neural networks for my own research as a graduate student. They motivated me to look for abstract principles that could be as simple as the laws of physics, but could explain biological as well as artificial intelligence. I'm truly delighted for them and for our field.
The Nobel Prize
@NobelPrize
BREAKING NEWS
The Royal Swedish Academy of Sciences has decided to award the 2024 #nobelprize in Physics to John J. Hopfield and Geoffrey E. Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks.” -
Pieter Abbeel reposted
Cade Gordon
@CadeGordonML
Excited to announce our new work! 🧬 Some highlights are:
- sequences likelihoods predict zero-shot fitness capabilities
- a new method to calculate pLM likelihood in O(1) instead of O(L) forward passes
- providing a causal between training data and outputs
- suggesting a new finetuning method to improve pLM capabilities
(1/9)
www.biorxiv.org/content/10.1101/2024.10.03.616542v1 -
Pieter Abbeel reposted
Boris Belousov
@_bbelousov
Presenting today at #icra40
MuJoCo MPC for Humanoid Control: Evaluation on HumanoidBench
Come to learn more at 14:15
We tackle #humanoidbench (humanoid-bench.github.io) with #mpc
Paper: arxiv.org/abs/2408.00342
Code: github.com/MoritzMeser/mujoco_mpc
Video:
Boris Belousov
@_bbelousov
To every action, there is always opposed an equal reaction
-
Pieter Abbeel reposted
Himanshu Gaurav Singh
@Cinnabar233
How can you scale up robot learning?
We introduce HOP: Extract hand-object trajectories from in-the-wild videos. Train a next-token-prediction model to get a manipulation prior. Adapt to your task using BC or RL. Checkout our webpage: hgaurav2k.github.io/hop/ More details in 🧵 -
Pieter Abbeel reposted
Peter Stone
@PeterStone_TX
10 years after DQN, what are deep RL’s impacts on robotics? Which robotic problems have seen the most thrilling real-world successes thanks to DRL? Where do we still need to push the boundaries, and how?
Our latest survey explores these questions! Read on for more details. 👇 -
Pieter Abbeel reposted
Philipp Wu
@philippswu
Really nice article from @ericjang about actuators in robotics. Great to see @1x_tech place such an emphasis on design. Safety through compliance and backdrivability were some of the core themes behind our BLUE project back in the day (berkeleyopenrobotics.github.io). Some thoughts 👇
Eric Jang
@ericjang11
Two robot arms move at the same speed, driven by different actuators with the same mass. The first arm collides with a table with a gentle tap. The second arm collides with the table, destroying both arm and table. Read this blog post to see why! 🦾💥
evjang.com/2024/08/31/motors.html -
Pieter Abbeel reposted
Antispace
@antispaceAI
Introducing Antispace, the first AIOS.
Antispace is a new gamified workspace that connects all your digital tools into an evolving HUD, maximizing freedom and eliminating context switching.
Each Antispace user can design their dedicated AI assistant, build personal memories together. It's not the destination but the memories we make along the way.
Antispace's proprietary Action-Oriented AI, allows you to automate tasks across all the tools in your space. You can even cross-coordinate simple actions to have a new emergent behavior.
We are so excited to see what you'll do with it.
Join Antispace today at www.antispace.sh -
Pieter Abbeel reposted
AIX Ventures
@aixventureshq
Congrats to @chingunkh, @indiegoman, & the entire @antispaceAI team on your launch. Here we go!
Antispace
@antispaceAI
Introducing Antispace, the first AIOS.
Antispace is a new gamified workspace that connects all your digital tools into an evolving HUD, maximizing freedom and eliminating context switching.
Each Antispace user can design their dedicated AI assistant, build personal memories together. It's not the destination but the memories we make along the way.
Antispace's proprietary Action-Oriented AI, allows you to automate tasks across all the tools in your space. You can even cross-coordinate simple actions to have a new emergent behavior.
We are so excited to see what you'll do with it.
Join Antispace today at www.antispace.sh -
Pieter Abbeel
@pabbeel
Amazing work by @ideogram_ai team. 80/20 rule, but in reverse: this team is willing (and exceptionally capable!) to put the extra 80% ingenuity and effort to nail that last 20% so you get what you really want!
And proud PhD advisor moment for myself, congratulations @hojonathanho!!!
Ideogram
@ideogram_ai
Introducing Ideogram 2.0 — our most advanced text-to-image model, now available to all users for free.
Today’s milestone launch also includes the release of the Ideogram iOS app, the beta version of the Ideogram API, and Ideogram Search.
Here’s what’s new… 🧵 -
Pieter Abbeel reposted
Teodora P L
@toolstelegraph
Ideogram 2.0 is here, and it comes with crazy realism .
Thread with image examples and prompts below 👇
Ideogram
@ideogram_ai
Introducing Ideogram 2.0 — our most advanced text-to-image model, now available to all users for free.
Today’s milestone launch also includes the release of the Ideogram iOS app, the beta version of the Ideogram API, and Ideogram Search.
Here’s what’s new… 🧵 -
Pieter Abbeel reposted
Remi Cadene
@RemiCadene
The wait is finally over!!! 😁
We just dropped an in-depth tutorial on how to build your own robot!
Teach it new skills by showing it a few moves with just a laptop.
Then watch your homemade robot act autonomously 🤯
1/🧵👇This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Carlo Sferrazza
@carlo_sferrazza
Can we make Transformers better and more efficient for robot learning?
Excited to introduce Body Transformer (BoT), an architecture that leverages robot embodiment in the attention mechanism, by treating it as a graph of sensors and actuators. -
Pieter Abbeel reposted
Philipp Wu
@philippswu
Heading to the @RL_Conference to present our work Semi-Supervised One Shot Imitation Learning 🏹 More details below!
Looking forward to catching up and chatting with old friends and new, let me know if you’re around! #rlc2024 -
Pieter Abbeel reposted
Amy Lu
@amyxlu
1/ 🧬 Excited to share CHEAP, our new work on compressed protein embeddings. We characterize the joint distribution of p(sequence, structure) in ESMFold's latent space, and find cool tidbits on compressibility, tokenizability, and pathologies:
www.biorxiv.org/content/10.1101/2024.08.06.606920v1.full.pdf
🧵 -
Pieter Abbeel reposted
Covariant
@CovariantAI
These are the AI startups to watch in 2024 - @Bloomberg
We're honored to be featured as a leading company pioneering AI for the physical world.
www.bloomberg.com/features/2024-top-ai-startups -
Pieter Abbeel
@pabbeel
5 tenure-track/tenured faculty positions in AI, Inequality, and Society at @UCBerkeley! Note the unusual, early application deadline: Sep 16.
eecs.berkeley.edu/connect/faculty-jobs/ -
Pieter Abbeel
@pabbeel
RT @antgoldbloom: .@benhamner and I have been quietly working on our newco, Sumble. We are building a knowledge graph for the world's exter…
-
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
I tried doing IK for a shadow hand the first year of my PhD and utterly failed. Took me 5 minutes to set it up with my new diff IK library and it works out of the box at > 1 kHZ.
-
Pieter Abbeel reposted
Younggyo Seo
@younggyoseo
Excited to present RSP: Representation learning with Stochastic frame Prediction, a new method that learns image representations from videos by training stochastic frame prediction model 🖼️
#icml2024
Paper: arxiv.org/abs/2406.07398 -
Pieter Abbeel reposted
Carlo Sferrazza
@carlo_sferrazza
Just released five robotics simulation environments we have been using to train reinforcement learning from vision and touch! Very easy to install. Powered by @GoogleDeepMind MuJoCo, Gymnasium, and robosuite.
GitHub link: github.com/carlosferrazza/tactile_envs -
Pieter Abbeel reposted
Kevin Frans
@kvfrans
Flow-matching implementation:
github.com/kvfrans/jax-flow
Flow-matching is very similar to diffusion, but simplifies things. Noised images are linear interpolations between (data, noise) pairs, and the network predicts *velocity* of this trajectory. -
Pieter Abbeel reposted
Covariant
@CovariantAI
“Covariant’s technology has absolutely given us an edge. When prospective clients ask us about our ability to handle unexpected spikes, our answer is twofold — elite tech combined with an incredible team of ops specialists and engineers.” - Karim Fofana, COO at Capacity
This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Kevin Frans
@kvfrans
Here is a simple VQVAE/VQGAN implementation in JAX. It supports codebook quantization, and also finite scalar quantization (FSQ) bottlenecks. You can also just use a regular VAE bottleneck.
github.com/kvfrans/jax-vqvae-vqgan
We are able to reproduce the original FID numbers! -
Pieter Abbeel reposted
Kevin Frans
@kvfrans
FID computation can be quite esoteric, here's a simple helper to do it in JAX. You can compute FID online during training! This implementation can closely match the numbers from OpenAI's guided-diffusion evaluations.
Code: github.com/kvfrans/jax-fid-parallel
. -
Pieter Abbeel reposted
Covariant
@CovariantAI
Trained on millions of picks from Covariant-powered robots worldwide, the Covariant Brain helps @KNAPP_AG_de's Pick-it-Easy robot recognize objects in totes and calculate grip points, learning and improving as it goes.
-
Pieter Abbeel reposted
Blake Resnick
@Blake_Resnick_
Today @BrincDrones has released the product I’ve wanted to build since I started the company.
This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Jim Fan
@DrJimFan
AlphaFold-3 is out, the latest iteration of the greatest breakthrough in AI for biology. What's new is that AlphaFold-3 uses diffusion to "render" the molecular structure. It starts from a fuzzy cloud of atoms and then materializes the molecule gradually through denoising.
We live on a timeline where learnings from Llama and Sora can inform and accelerate life sciences. I find this level of generality absolutely mind-boggling. The same transformer+diffusion backbone that generates fancy pixels can also imagine proteins, as long as you convert the data to sequences of floats accordingly.
We are not there yet at a single AGI model, but we have successfully built a menu of general-purpose AI recipes that transfer training, data, and neural architectures across domains. This should not work, but thank god it does! -
Pieter Abbeel reposted
Philipp Wu
@philippswu
New work with @YideShentu! We present Latent Code as Bridges🌉 or LCB. We leverage LLMs for robotics by finetuning the LLM to output latent embeddings to control the lower level policy. This hierarchical approach enables end2end training while maintaining the advantages of LLMs!
-
Pieter Abbeel reposted
Alex Kendall
@alexgkendall
Chuffed to host @RishiSunak and @michelledonelan at @wayve_ai today to celebrate our Series C news and talk all about Embodied AI!
-
Pieter Abbeel reposted
Sherry Yang
@sherryyangML
Happy to share that UniSim universal-simulator.github.io was selected for an Outstanding Paper Award at #iclr2024.
Check out the oral presentation today at 10:30am Oral 1B and poster on Wed at 4:30-6:30pm #87.
Thanks to the award committee @eunsolc, @katjahofmann, @liu_mingyu, @nanjiang_cs, @guennemann, @optiML, @tkipf, @CevherLIONS and my amazing collaborators @du_yilun,@coolboi95,@JonathanTompson, Leslie Kaelbling, Dale Schuurmans, @pabbeel.
ICLR
@iclr_conf
Announcing the #iclr2024 Outstanding Paper Awards:
blog.iclr.cc/2024/05/06/iclr-2024-outstanding-paper-awards/
Shoutout to the awards committee: @eunsolc, @katjahofmann, @liu_mingyu, @nanjiang_cs, @guennemann, @optiML, @tkipf, @CevherLIONS -
Pieter Abbeel reposted
vint
@vint_lee_
I'll be presenting DreamSmooth at #iclr2024! 😊
Reward prediction is vital to model-based RL. Yet, many algorithms struggle to predict sparse rewards accurately. By addressing this, DreamSmooth dramatically improves task performance in many sparse-reward environments.
🧵👇 -
Pieter Abbeel reposted
Kuba
@kuba_AI
"Better than what's known" - the dream of AI in model-based optimization. Excited to present my paper with @pabbeel & @svlevine on Functional Graphical Models (arxiv.org/pdf/2401.05442) giving theoretical foundations for chasing that dream, at @aistats_conf ! #aistats_conf #ai
-
Pieter Abbeel
@pabbeel
A fun short lecture on parallelization for training large models:
youtu.be/Hr2FWHBuNXs
by @wilson1yan and @haoliuhl -
Pieter Abbeel reposted
Charles C Onu
@onucharlesc
Guess what app is ranked #21 in the medical category on the app store 🤗
-
Pieter Abbeel reposted
bjorn verbrugghe
@bjorn1981
Geschiedenis van #ai : term in 1956, tot 2012 was er niet dat werkte. In 2012 was er alexnet en #deeplearning . Hoe meer data, hoe beter het werkt @pabbeel
-
Pieter Abbeel
@pabbeel
Guest Lecture on Neural Radiance Fields by the great @BenMildenhall himself!
youtu.be/Z96YMktT_T4
Pieter Abbeel
@pabbeel
Full course on Deep Unsupervised Learning:
sites.google.com/view/berkeley-cs294-158-sp24/home -
Pieter Abbeel
@pabbeel
Great guest lecture yesterday by John Ingraham on Generative Models for Science
youtu.be/4dyf_SALtxM
Pieter Abbeel
@pabbeel
full course:
sites.google.com/view/berkeley-cs294-158-sp24/home -
Pieter Abbeel reposted
Peter Chen
@peterxichen
What separates lab robot demos and robots in production? Extremely high reliability. This requires our model to robustly handle many long-tail scenarios like the one in attached picture, where one stray barcode label can tank the 99.95% sortation accuracy requirements. (1/n)
Pieter Abbeel
@pabbeel
resonates with my own experience:
robot demo --> INSANE PAIN --> robot creating value in production
luckily I am hopeful we have by now paid most of our dues at Covariant :) -
Pieter Abbeel
@pabbeel
resonates with my own experience:
robot demo --> INSANE PAIN --> robot creating value in production
luckily I am hopeful we have by now paid most of our dues at Covariant :) -
Pieter Abbeel reposted
Covariant
@CovariantAI
RFM-1's latest scaling update enables robots with in-context learning of grasping improvements.
The video shows the self-reflective reasoning capability — after a few tries and failing, the robot has an internal dialogue, hypothesizing that its current gripper is not suited for fabric. Then it comes up with a new strategy and picks up the socks by the paper label instead. -
Pieter Abbeel reposted
Covariant
@CovariantAI
Building a truly general AI for robotics must be done in the real world. To make this happen, our focus lies in addressing critical issues and delivering true value to our customers.
During the recent #nvidiagtc2024, @peterxichen highlighted how generative AI is reshaping the robotics landscape.
@NVIDIARobotics -
Pieter Abbeel reposted
Ken Goldberg
@Ken_Goldberg
"Why don't we have better robots yet?": just posted on the @TEDTalks home page under Newest Talks (3rd row from the top) with links to @PieRobotics and @Forbes article on art by Ben Wolff @creativecellist @Berkeley_AI @UCBerkeley @Cal_Engineer ted.com
-
Pieter Abbeel reposted
Charles C Onu
@onucharlesc
Guess, what?! Nanni AI has now translated 1 million infant cries globally 🎉 🎉 🎉 That's a lot of "waaaahs" and a huge milestone for us.
We're giving voice to infants all over the world and empowering parents to provide better and rapid care for their little ones. Onward to the next milestone...50M, 100M...Stay tuned!
The Nanni app is now used in every country in the world and is available on all app stores. Share with your friends, family and foes.
#nanni #nanniai -
Pieter Abbeel reposted
Peter Chen
@peterxichen
What makes the training data for RFM-1 unique? A few properties that are distinctive from typical lab data: 1. real-world complexity: picking from extremely cluttered scenes where item occlusion presents a challenge for reliability. 2. high-speed handling: the dynamics of low-speed manipulation is a lot easier to manage, e.g. one doesn't need to worry about scenarios like pulling an item at the bottom will scoop items above or items sliding around between picks.
Covariant
@CovariantAI
This is not a one-off cycle. This is performance over repeated cycles – the new benchmark for reliable AI Robotic systems.
High pick rates, navigating cluttered environments, no double-picks, scoops, or errors – just consistent flawless execution, parcel after parcel.This media couldn't be displayed. See it on the original post. -
Pieter Abbeel
@pabbeel
GTC 5 years ago! Jensen without his signature jacket 🙃
-
Pieter Abbeel reposted
Xiaolong Wang
@xiaolonw
This project looks super fun! And it is also wonderful to see TD-MPC2 works so well on this benchmark compared to PPO/SAC/Dreamerv3!
Carlo Sferrazza
@carlo_sferrazza
Humanoids 🤖 will do anything humans can do. But are state-of-the-art algorithms up to the challenge?
Introducing HumanoidBench, the first-of-its-kind simulated humanoid benchmark with 27 distinct whole-body tasks requiring intricate long-horizon planning and coordination.
🧵👇 -
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
An awesome and refreshing benchmark by Carlo and collaborators featuring practically useful tasks like truck unloading. All open-source, built on MuJoCo and accelerated with JAX!
Carlo Sferrazza
@carlo_sferrazza
Humanoids 🤖 will do anything humans can do. But are state-of-the-art algorithms up to the challenge?
Introducing HumanoidBench, the first-of-its-kind simulated humanoid benchmark with 27 distinct whole-body tasks requiring intricate long-horizon planning and coordination.
🧵👇 -
Pieter Abbeel
@pabbeel
Era of humanoids is coming, but not always easy to afford one or run real world experiments. Luckily there are also many many research challenges that can be investigated in simulation, join the fun, try to get your AI to do well on #humanoidbench!
Carlo Sferrazza
@carlo_sferrazza
Humanoids 🤖 will do anything humans can do. But are state-of-the-art algorithms up to the challenge?
Introducing HumanoidBench, the first-of-its-kind simulated humanoid benchmark with 27 distinct whole-body tasks requiring intricate long-horizon planning and coordination.
🧵👇 -
Pieter Abbeel reposted
Rocky Duan
@rocky_duan
Infra behind RFM-1: Our Robotics Foundation Model is trained on several clouds; this gives us more GPUs for high experimentation velocity. Orchestration across clouds & serving entirely managed by @skypilot_org , lifting lots of infra burden off our team. More improvements soon!
-
Pieter Abbeel reposted
TechCrunch
@TechCrunch
This week Covariant announced the launch of RFM-1 (Robotics Foundation Model 1).
RFM-1 is the result of, among other things, a massive trove of data collected from the deployment of Covariant’s Brain AI platform.
Read more on Covariant here: tcrn.ch/3VgtD3l -
Pieter Abbeel
@pabbeel
From @mckbrando, first author of Apple's MM1. Extra excited to teach CS188 Intro to AI again this Fall!
Also, MM1 paper link:
arxiv.org/abs/2403.09611
And Brandon's homepage:
mckinziebrandon.me -
Pieter Abbeel reposted
Berkeley AI Research
@berkeley_ai
Looking to hire top AI talent?
We've compiled a list of the brilliant @berkeley_ai Ph.D. Graduates of 2024 who are currently on the academic and industry job markets. (Thanks to our friends @StanfordAILab for the idea!)
Check it out here:
bair.berkeley.edu/blog/2024/03/11/grads-2024/ -
Pieter Abbeel reposted
Index Ventures
@IndexVentures
.@CovariantAI announces a truly groundbreaking achievement. With the release of RFM-1: a Robotics Foundation Model that provides robots the human-like ability to reason. 🔗 Read more on this advancement: www.wired.com/story/quest-ai-chatbots-a-hand-arm-robots/
-
Pieter Abbeel
@pabbeel
Really nice deeper dive on Covariant's RFM-1 by @IEEESpectrum 's @BotJunkie:
spectrum.ieee.org/covariant-foundation-model -
Pieter Abbeel
@pabbeel
“The groups which are going to train good models are going to be the ones that have either access to already large amounts of robot data or capabilities to generate those data,” -- Lerrel Pinto [couldn't agree more @LerrelPinto!]
Great in-depth article on Covariant's RFM-1 in @techreview by James O'Donnell
www.technologyreview.com/2024/03/11/1089653/an-openai-spi… -
Pieter Abbeel
@pabbeel
"Peter Chen, CEO of the robot software company Covariant, sits in front of a chatbot interface resembling the one used to communicate with ChatGPT. "Show me the tote in front of you," he types. In reply ... "
Really nice article on Covariant's RFM-1 by @willknight in @WIRED -
Pieter Abbeel
@pabbeel
"RFM-1 is the result of, among other things, a massive trove of data collected from the deployment of Covariant’s Brain AI platform."
Very nice article by Brian Heater on @TechCrunch
techcrunch.com/2024/03/11/covariant-is-building-chatgpt-f… -
Pieter Abbeel reposted
Sunil Dhaliwal
@dhaliwas
The best domain-specific models leverage massive amounts of proprietary data.
Latest example: RFM-1 by @CovariantAI was built on the world's largest (and fastest-growing) dataset of real-world robotic deployments.
Best data = best model performance.
www.nytimes.com/2024/03/11/technology/ai-robots-technolog… -
Pieter Abbeel reposted
Jim Fan
@DrJimFan
One of the very few robot models trained on years of in-the-wild real data, instead of toy benchmarks in lab settings. We need more data flywheels in the physical world!
Congrats @pabbeel @peterxichen @rocky_duan and @CovariantAI team!
Pieter Abbeel
@pabbeel
Yes, 2024 is shaping up a big year for robotics!
Introducing @CovariantAI's RFM-1, which just like Sora can generate video, but RFM-1 does it for robotic interaction with the world.
But there is so much more it can do. RFM-1 is a multimodal any-to-any sequence model. RFM-1 tokenizes 5 modalities: video, keyframes, text, sensory readings, robot actions. Any-to-any means RFM-1 can handle any subset on the input and any subset on the output. -
Pieter Abbeel
@pabbeel
Accelerating robotics development!
Foxglove
@foxglove
✨ Announcing Foxglove 2.0 ✨
We’re setting a new standard in robotics observability by making it faster, easier, and cheaper to understand how your machines sense, think, and act:
🤝 Streamlined UI for smoother robotics observability
📡 Automatic data offload through Foxglove Agent
💳 Updated pricing plans to make Foxglove accessible for teams of all sizes
🔎 Changes to our open-source strategy
Read all the Foxglove 2.0 details here: foxglove.dev/blog/foxglove-2-0-unifying-robotics-observab… -
Pieter Abbeel reposted
Radical Ventures
@radicalvcfund
Covariant's Robotics Foundation Model RFM-1 represents a remarkable leap forward toward building generalized AI models that can accurately simulate and operate in the demanding conditions of the physical world.
Covariant
@CovariantAI
Today, we are introducing RFM-1, our Robotics Foundation Model giving robots human-like reasoning capabilities.
-
Pieter Abbeel reposted
Peter Chen
@peterxichen
At Covariant, we have been deploying robots to the real world and thinking hard about how to build truly AI for general purpose robots that can go beyond demos. For a long time, we are building up large robotics datasets through our deployments without the right model that can absorb all of the bits. And now that finally changed.(1/n) www.nytimes.com/2024/03/11/technology/ai-robots-technolog…
-
Pieter Abbeel reposted
Alex Kendall
@alexgkendall
Exciting result showing how robust, accessible and trustworthy robotics is becoming with AI foundation models. And I'm sure lots more to come.. congratulations @pabbeel @peterxichen 🎉
Covariant
@CovariantAI
Today, we are introducing RFM-1, our Robotics Foundation Model giving robots human-like reasoning capabilities.
-
Pieter Abbeel reposted
WIRED
@WIRED
Robotics startup Covariant is experimenting with a ChatGPT-style chatbot that can control a robotic arm, as a way to create machines that can be more helpful in the physical world. wired.trib.al/e9cGfad
-
Pieter Abbeel reposted
Peter Zhenghao Qian
@peterzhqian
Very excited to share what we’ve been working on - robots that can understand language and physics and get things done in the real world!
Pieter Abbeel
@pabbeel
Yes, 2024 is shaping up a big year for robotics!
Introducing @CovariantAI's RFM-1, which just like Sora can generate video, but RFM-1 does it for robotic interaction with the world.
But there is so much more it can do. RFM-1 is a multimodal any-to-any sequence model. RFM-1 tokenizes 5 modalities: video, keyframes, text, sensory readings, robot actions. Any-to-any means RFM-1 can handle any subset on the input and any subset on the output. -
Pieter Abbeel reposted
Amplify Partners
@AmplifyPartners
Today @CovariantAI released RFM-1: a Robotics Foundation Model that provides robots the human-like ability to reason. This represents the first time Generative AI has successfully given commercial robots a deeper understanding of language and the physical world!
WIRED
@WIRED
Robotics startup Covariant is experimenting with a ChatGPT-style chatbot that can control a robotic arm, as a way to create machines that can be more helpful in the physical world. wired.trib.al/e9cGfad
-
Pieter Abbeel reposted
Rocky Duan
@rocky_duan
Very excited to share what our research team at Covariant has been working on: RFM-1, our latest Robotics Foundation Model. Built on top of high-quality multimodal data the Covariant robotics fleet has collected over the years, this project embodies our commitment to pushing the boundaries of how AI can interact with and understand the physical world. We're eager to share this milestone with the community and invite feedback and collaboration. Read more at covariant.ai/insights/introducing-rfm-1-giving-robots-hum…
-
Pieter Abbeel reposted
Abhishek Gupta
@abhishekunique7
Exciting to see what @pabbeel, Anusha Nagabandi, @clavera_i, @CarlosFlorensa, Nikhil Mishra and other friends at covariant have been up to!
Covariant
@CovariantAI
Today, we are introducing RFM-1, our Robotics Foundation Model giving robots human-like reasoning capabilities.
-
Pieter Abbeel
@pabbeel
Covariant's RFM-1 as covered by @nytimes, @BusinessWire, @TechCrunch, @WIRED, @techreview
www.nytimes.com/2024/03/11/technology/ai-robots-technolog…
www.businesswire.com/news/home/20240311948570/en/Covarian…
techcrunch.com/2024/03/11/covariant-is-building-chatgpt-f…
www.wired.com/story/quest-ai-chatbots-a-hand-arm-robots/
www.technologyreview.com/2024/03/11/1089653/an-openai-spi…
🙏🚀🙏🚀🙏🚀🙏🚀🙏
Pieter Abbeel
@pabbeel
Yes, 2024 is shaping up a big year for robotics!
Introducing @CovariantAI's RFM-1, which just like Sora can generate video, but RFM-1 does it for robotic interaction with the world.
But there is so much more it can do. RFM-1 is a multimodal any-to-any sequence model. RFM-1 tokenizes 5 modalities: video, keyframes, text, sensory readings, robot actions. Any-to-any means RFM-1 can handle any subset on the input and any subset on the output. -
Pieter Abbeel
@pabbeel
Yes, 2024 is shaping up a big year for robotics!
Introducing @CovariantAI's RFM-1, which just like Sora can generate video, but RFM-1 does it for robotic interaction with the world.
But there is so much more it can do. RFM-1 is a multimodal any-to-any sequence model. RFM-1 tokenizes 5 modalities: video, keyframes, text, sensory readings, robot actions. Any-to-any means RFM-1 can handle any subset on the input and any subset on the output.Post not available here, view on X
Pieter Abbeel
@pabbeel
Btw, this is a sampling of what our Covariant robots look like in action:
Pieter Abbeel
@pabbeel
To wrap up, a quick explainer video on RFM-1 video generation capabilities and use cases
This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Raluca Ada Popa
@ralucaadapopa
Last chance to submit a talk for presentation at the Confidential Computing Summit! Submit about your tech or use cases pertaining to confidential data or AI, trustworthy generative AI, or enclaves. Deadline extended to Friday March 8, hard deadline.
www.confidentialcomputingsummit.com/call-for-speakers @confcompsummit -
Pieter Abbeel reposted
Fangchen Liu
@fangchenliu_
Can we leverage VLMs for robot manipulation in the open world? Checkout our new work MOKA, a simple and effective visual prompting method!
-
Pieter Abbeel reposted
Kuan Fang
@KuanFang
Can we harness the emergent capabilities of VLMs to solve robotic manipulation tasks in zero-shot and few-shot manners?
We introduce MOKA, which employs pre-trained VLMs to predict affordances and generate motions through mark-based visual prompting.
moka-manipulation.github.io -
Pieter Abbeel reposted
Toru
@ToruO_O
Achieving bimanual dexterity with RL + Sim2Real!
toruowo.github.io/bimanual-twist/
TLDR - We train two robot hands to twist bottle lids using deep RL followed by sim-to-real. A single policy trained with simple simulated bottles can generalize to drastically different real-world objects. -
Pieter Abbeel reposted
Jitendra MALIK
@JitendraMalikCV
Another success of sim-to-real for training robot policies! This task, using two multi-fingered hands, requires considerable dexterity, and is hopefully representative of other household tasks that we wish to solve in the future.
Toru
@ToruO_O
Achieving bimanual dexterity with RL + Sim2Real!
toruowo.github.io/bimanual-twist/
TLDR - We train two robot hands to twist bottle lids using deep RL followed by sim-to-real. A single policy trained with simple simulated bottles can generalize to drastically different real-world objects. -
Pieter Abbeel
@pabbeel
More coverage for Large World Model! :)
bnnbreaking.com/world/us/uc-berkeley-students-ai-model-ou…
Pieter Abbeel
@pabbeel
paper, model, etc here:
largeworldmodel.github.io -
Pieter Abbeel reposted
TollBit
@TollbitOfficial
Leading media and tech investors bet on @tollbitofficial - the first platform to tackle the new economics of content in the ai era. Story here www.axios.com/newsletters/axios-pro-rata-53fd2652-ac31-41… @axios @danprimack. #ai #publishing
-
Pieter Abbeel reposted
Kevin Frans
@kvfrans
Functional Reward Encodings (FRE) provide a simple, unsupervised approach to offline zero-shot RL:
1. Learn a robust latent space of reward functions, then
2. Train a generalist agent on *random functions*, such that new tasks naturally map to solutions.
kvfrans.com/fre/ -
Pieter Abbeel reposted
Electric Sheep
@sheeprobotics
Animal-like robots are the future.
Introducing Verdie, a physical agent for sustainable outdoor maintenance capable of learning multiple power tools.
Verdie was trained in the @nvidia Isaac simulator for just 5 hours before string trimming lawns for the first time.
Verdie is being deployed in 2024 to work alongside our service crews to automate many different dull, dangerous and dirty tasks.This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Sherry Yang
@sherryyangML
Video generation will revolutionize decision making in the physical world like how language models have changed the digital world.
Interested in the implications of video generation models like UniSim and Sora? Check out our position paper:
arxiv.org/abs/2402.17139 -
Pieter Abbeel reposted
AK
@_akhaliq
Video as the New Language for Real-World Decision Making
Both text and video data are abundant on the internet and support large-scale self-supervised learning through next token or frame prediction. However, they have not been equally leveraged: language models have had significant real-world impact, whereas video generation has remained largely limited to media entertainment. Yet video data captures important information about the physical world that is difficult to express in language. To address this gap, we discuss an under-appreciated opportunity to extend video generation to solve tasks in the real world. We observe how, akin to language, video can serve as a unified interface that can absorb internet knowledge and represent diverse tasks. Moreover, we demonstrate how, like language models, video generation can serve as planners, agents, compute engines, and environment simulators through techniques such as in-context learning, planning and reinforcement learning. We identify major impact opportunities in domains such as robotics, self-driving, and science, supported by recent work that demonstrates how such advanced capabilities in video generation are plausibly within reach. Lastly, we identify key challenges in video generation that mitigate progress. Addressing these challenges will enable video generation models to demonstrate unique value alongside language models in a wider array of AI applications. -
Pieter Abbeel
@pabbeel
Gave a lecture on Diffusion Models yesterday with @wilson1yan and @kvfrans
youtu.be/DsEDMjdxOv4
Thanks to it being Lecture 6 into the semester (rather than Lecture 1), we could even include Sora, Mobile Diffusion, SDXL-Lightning :)Post not available here, view on X
Pieter Abbeel
@pabbeel
If you want to dive even deeper, stay tuned, HW on Diffusion coming out in ~2weeks!
Pieter Abbeel
@pabbeel
All course materials of the Deep Unsupervised Learning class here:
sites.google.com/view/berkeley-cs294-158-sp24/home
So far covered Autoregressive Models, Flow Models, VAEs, GANs, Diffusion Models -
Pieter Abbeel reposted
Xingyu Lin
@Xingyu2017
What state representation should robots have? 🤖 I’m thrilled to present an Any-point Trajectory Model (ATM), which models physical motions from videos without additional assumptions and shows significant positive transfer from cross-embodiment human and robot videos! 🧵👇
-
Pieter Abbeel reposted
Covariant
@CovariantAI
Robot reels on repeat.
The new go-to place to experience AI robots in action – from Kentucky warehouses to Berlin, and the latest research emerging from our California-based lab. covariant.ai/insights/reels/ -
Pieter Abbeel reposted
Jiquan Ngiam
@JiquanNgiam
Large world models by @haoliuhl et al. is worth checking out if you like the Gemini-1.5 and Sora results. It has 1M context window, generates videos, and is open sourced.
1. LWM shows high recall over 1M context window, and performs exceptionally well with video chat as well.
2. It's able to generate video from text, with camera pans, and high degree of coherency.
3. Cleverly leverages Ring Attention with distributed computing; hosts are responsible a block of Q, and pass along KV blocks. This works particularly well with the TPU torus interconnects.
Would love to see the community scale this up.
largeworldmodel.github.io
cc @wilson1yan @matei_zaharia @pabbeel -
Pieter Abbeel reposted
Ken Goldberg
@Ken_Goldberg
The @OpenAI core team behind #sora, Tim Brooks (@_tim_brooks) and Bill Peebles (@billpeeb), are recent graduates of @UCBerkeley @berkeley_ai BAIR Lab.
Apparently they chose the name Sora because it "evokes the idea of limitless creative potential". -
Pieter Abbeel reposted
Ken Goldberg
@Ken_Goldberg
#1 topic requested by @ycombinator is #robotics: "Robotics hasn't yet had its GPT moment, but we think it’s close.... We're interested in people building software tools to help other people to make robots: www.ycombinator.com/blog/ycs-latest-request-for-startups
-
Pieter Abbeel
@pabbeel
Current works are restricted to short sequences of texts and images, limiting their ability to model the world.
Presenting Large World Model (LWM): capable of processing long text, images, videos of over 1M tokens (and *no* lost in the middle!)
Project: largeworldmodel.github.io
Hao Liu
@haoliuhl
We are excited to share Large World Model (LWM), a general-purpose 1M context multimodal autoregressive model. It is trained on a large dataset of diverse long videos and books using RingAttention, and can perform language, image, and video understanding and generation.
-
Pieter Abbeel reposted
Hao Liu
@haoliuhl
We open-sourced a family of 7B parameter models capable of processing long text documents (LWM-Text, LWM-Text-Chat) and videos (LWM, LWM-Chat) of over 1M tokens, along with the codebase for training and inference. The models are available at huggingface.co/LargeWorldModel.
-
Pieter Abbeel reposted
Hao Liu
@haoliuhl
We are excited to share Large World Model (LWM), a general-purpose 1M context multimodal autoregressive model. It is trained on a large dataset of diverse long videos and books using RingAttention, and can perform language, image, and video understanding and generation.
-
Pieter Abbeel reposted
AIX Ventures
@aixventureshq
Yesterday we announced our $202M Fund II to back early-stage startups. Today we want to thank the entire AIX Ventures team for their hard work.
@pabbeel, @antgoldbloom, @shaunbjohnson, @shyammani_, @chrmanning, Christina Sfakianos, @RichardSocher, @saratatman -
Pieter Abbeel reposted
Aran Komatsuzaki
@arankomatsuzaki
World Model on Million-Length Video And Language With RingAttention
Open-sources 7B models capable of processing long text documents and videos of over 1M tokens
proj: largeworldmodel.github.io
abs: arxiv.org/abs/2402.08268 -
Pieter Abbeel reposted
Matei Zaharia
@matei_zaharia
Super cool work by @haoliuhl and @wilson1yan using Ring Attention. A multimodal 7B model with 1M context length.
Aran Komatsuzaki
@arankomatsuzaki
World Model on Million-Length Video And Language With RingAttention
Open-sources 7B models capable of processing long text documents and videos of over 1M tokens
proj: largeworldmodel.github.io
abs: arxiv.org/abs/2402.08268 -
Pieter Abbeel reposted
AK
@_akhaliq
World Model on Million-Length Video And Language With RingAttention
paper page: huggingface.co/papers/2402.08268
Current language models fall short in understanding aspects of the world not easily described in words, and struggle with complex, long-form tasks. Video sequences offer valuable temporal information absent in language and static images, making them attractive for joint modeling with language. Such models could develop a understanding of both human textual knowledge and the physical world, enabling broader AI capabilities for assisting humans. However, learning from millions of tokens of video and language sequences poses challenges due to memory constraints, computational complexity, and limited datasets. To address these challenges, we curate a large dataset of diverse videos and books, utilize the RingAttention technique to scalably train on long sequences, and gradually increase context size from 4K to 1M tokens. This paper makes the following contributions: (a) Largest context size neural network: We train one of the largest context size transformers on long video and language sequences, setting new benchmarks in difficult retrieval tasks and long video understanding. (b) Solutions for overcoming vision-language training challenges, including using masked sequence packing for mixing different sequence lengths, loss weighting to balance language and vision, and model-generated QA dataset for long sequence chat. (c) A highly-optimized implementation with RingAttention, masked sequence packing, and other key features for training on millions-length multimodal sequences. (d) Fully open-sourced a family of 7B parameter models capable of processing long text documents (LWM-Text, LWM-Text-Chat) and videos (LWM, LWM-Chat) of over 1M tokens. This work paves the way for training on massive datasets of long video and language to develop understanding of both human knowledge and the multimodal world, and broader capabilities. -
Pieter Abbeel
@pabbeel
Since AlexNet/ImageNet moment in October 2012, stock multiples:
nvidia: ~250x
amd: ~80x
intel: ~2x -
Pieter Abbeel reposted
Nikhil Buduma
@nkbuduma
"Ambience Healthcare Rakes In $70M for Documentation Tech That One Customer Calls 'Close to Perfection'" -@medcitynews
MedCity News
@medcitynews
SaaS startup Ambience Healthcare raised $70 million from investors including OpenAI, Andreessen Horowitz, Optum Ventures and Kleiner Perkins. The San Francisco-based company offers tools to alleviate clinicians’ burnout by helping them with... medcitynews.com/2024/02/healthcare-clinical-documentation…
-
Pieter Abbeel
@pabbeel
RT @MurtyNag: AI in the physical world - Sharing a feature that NBC news ran on Electric Sheep @sheeprobotics today!
https://t.co/D3JezoO7… -
Pieter Abbeel reposted
Zhongyu Li
@ZhongyuLi4
Interested in making your bipedal robots to be athletes? We summarized our RL work to create robust & adaptive controllers for general bipedal skills. 400m-dash, running over terrains/against perturbations, targeted jumping, compliant walking, not a problem for bipeds now.🧵👇
This media couldn't be displayed. See it on the original post. -
Pieter Abbeel
@pabbeel
Great opportunity to come learn about latest advances at intersection of AI <> people
Berkeley AI Research
@berkeley_ai
This week in the Brilliance of Berkeley lecture series, we have BAIR faculty @ancadianadragan talking about "Robotics algorithms that take people into account"!
Register your interest to join in person or online: curricularconnections.berkeley.edu/ls110/
news.berkeley.edu/2024/01/18/new-brilliance-of-berkeley-c… -
Pieter Abbeel
@pabbeel
How to scan documents with iphone?
I didn't see it coming that the Notes app was the place to start or that it even was natively supported (thought I'd be recommended dedicated scanner apps). Works like a charm turns out, generates PDFs no less
www.perplexity.ai/search/how-to-scan-eKMhaTseQyajtC.6iov9… -
Pieter Abbeel reposted
Covariant
@CovariantAI
Thrilled to share that we've earned a spot on @a16z American Dynamism AI list, recognizing companies at the forefront of tackling pressing societal challenges and pushing the boundaries of AI innovation. a16z.com/american-dynamism-50-ai/
-
Pieter Abbeel reposted
Mohammad Norouzi
@mo_norouzi
Ideogram is hiring! Join us to build the future of AI for creativity!
Ideogram
@ideogram_ai
We're hiring a machine learning engineer excited about pushing the frontier of AI for creativity. You'll work directly with the the pioneers of denoising diffusion models. You can apply here if you're interested: jobs.ashbyhq.com/ideogram/557ee59e-aeef-4c06-b802-43657a3…
-
Pieter Abbeel reposted
Covariant
@CovariantAI
Back in 2017, when we founded Covariant, our key insight was that the future of AI and robotics lies in foundation models.
This week, @peterxichen joined @saranormous on the @NoPriorsPod to explore our pioneering work in AI for commercial robotics. -
Pieter Abbeel
@pabbeel
Love the content and how this is done! Great insights into how @berkeley_ai Prof. Mike Jordan thinks about the future of AI and also a look into his research style
www.youtube.com/watch?v=3lXphIYfoBM
And thank you Barbara Rosario for creating! -
Pieter Abbeel reposted
Talia Goldberg
@TaliaGold
In October 2022, I wrote a piece about a trillion dollar opportunity in search. @perplexity_ai is that company. I only wish I realized a year sooner ;) Better late than never.
Thrilled to officially be a part of the Perplexity journey.
Talia Goldberg
@TaliaGold
Google is the front page of the internet, but it's deteriorating. There is a trillion dollar opportunity to reimagine search.
The next Google will look nothing like Google
talia.gold/2022/10/31/a-trillion-dollar-opportunity/ -
Pieter Abbeel reposted
Aravind Srinivas
@AravSrinivas
Excited to announce we've raised 73.6M$ at 520M$ valuation, led by IVP, along with our seed and Series A lead investors NEA, Elad Gil, Nat Friedman. Honored to have NVIDIA, Jeff Bezos, @tobi, Databricks, @naval, @rauchg, @balajis partner in this round, among others.
-
Pieter Abbeel
@pabbeel
Biggest pro-tip I got in 2023 (h/t @RichardSocher):
Keeping two dumbbells right on my desk, for quick lifts in little dead moments between meetings, when listening to a seminar, etc.
Went from when will I ever have time to lift again to lifting multiple times every day -
Pieter Abbeel reposted
Covariant
@CovariantAI
Sending holiday cheer from our robots across the world!❄️
-
Pieter Abbeel reposted
Bianca Caban
@bianca_caban
Thanks for the shoutout, and for having me @inside!
#mof5
INSIDE
@inside
Bianca Caban (@bianca_caban) of @aixventureshq shared at Meet Our Fund 5 their commitment to early-stage AI innovation.
With the investment surge from $3B to $14B in GenAI startups, AIX sees a future where AI augments every knowledge worker. -
Pieter Abbeel reposted
Raphael Townshend
@raphaeljlt
Excited to see this out! Atomic was built from the ground up to fuse both cutting-edge experimental work and AI, and the ATOM-1 foundation model is a beautiful example of this in action.
Onwards and upwards to unlocking a whole new world of RNA-focused therapies!
#atom1 #rna
Atomic AI
@AtomicAICo
✴️ An LLM for RNA? ✴️
ATOM-1 is a first-of-its-kind #ai foundation model to enable the design of therapies across the #rna landscape.
Learn more: www.businesswire.com/news/home/20231215527488/en/Atomic-A… -
Pieter Abbeel reposted
Ajay Jain
@ajayj_
Having a blast in New Orleans! Crazy to think that we presented DDPM as a virtual poster just 3 years ago at NeurIPS 2020. In person food is much better 😋 Check out these beignets I made in a few minutes on Genmo.
Come find / DM me to chat about big diffusion models for video. I'm also hiring outstanding researchers and engineers to work closely with me on generative models and distributed systems. -
Pieter Abbeel reposted
Wilson Yan
@wilson1yan
1/ Excited to announce MoCA - a simple yet effective video editing method that accomplishes diverse spatial (style, background, object) and motion video edits using a motion conditioned image animation model.
-
Pieter Abbeel reposted
Aravind Srinivas
@AravSrinivas
Excited to announce that pplx-api is coming out of beta and moving to usage based pricing, along with the first-ever live LLM APIs that are grounded with web search data and have no knowledge cutoff! pplx.ai/online-llms
-
Pieter Abbeel reposted
Ideogram
@ideogram_ai
We just launched image prompting. Now you can upload an image and control the output using visual input in addition to text. This is available to all of our plus subscribers. Enjoy!
-
Pieter Abbeel
@pabbeel
. @nvidia Jensen Huang has always been a big supporter of the research we do at Berkeley AI Research lab and also back when we were at OpenAI. It was really nice to now get to show him at @CovariantAI what our Robotics Foundation models are starting to enable in the real world!
-
Pieter Abbeel reposted
Sam Toyer
@sdtoyer
Prompt injection is a huge security problem for LLM apps. To study this, we built Tensor Trust: a game where you create and defend against prompt injections. We’re releasing a paper + dataset with 70k unique attacks, 40k unique defense prompts, and new robustness benchmarks. 👉
-
Pieter Abbeel reposted
Turhan Can Kargın 🐳
@turhancan97
The podcasts I listen to most often this year. Thank you all for the excellent content. @therobotbrains @pabbeel @lexfridman @IncontrolP @twimlai
#spotifywrapped -
Pieter Abbeel
@pabbeel
Congratulations @rocky_duan, couldn't be more deserved!
Covariant
@CovariantAI
Rocky’s visionary research and leadership have delivered critical breakthroughs in the field of AI and Robotics. Congratulations @rocky_duan on 2024 @Forbes' 30 Under 30!
-
Pieter Abbeel reposted
Sherry Yang
@sherryyangML
Checkout UniMat -- a unified representation of materials that enables scaling of diffusion models to millions of stable crystal structures.
Website: unified-materials.github.io
Paper: arxiv.org/abs/2311.09235 -
Pieter Abbeel reposted
RL_Conference
@RL_Conference
Thrilled to announce the first annual Reinforcement Learning Conference @RL_Conference, which will be held at UMass Amherst August 9-12!
RLC is the first strongly peer-reviewed RL venue with proceedings, and our call for papers is now available: rl-conference.cc. -
Pieter Abbeel reposted
Covariant
@CovariantAI
“The “GPT for robotics” is being built the same way as GPT was — laying the groundwork for a revolution that will, yet again, redefine AI as we know it.” - @peterxichen @TechCrunch
techcrunch.com/2023/11/10/ai-robotics-gpt-moment-is-near/ -
Pieter Abbeel reposted
Covariant
@CovariantAI
1/2 Imagine a robot that can learn and adapt effortlessly, regardless of what it encounters. This is the fundamental challenge in AI for robotics.
@peterxichen @twistartups @Jason -
Pieter Abbeel reposted
Bianca Caban
@bianca_caban
Had a blast speaking about @aixventureshq & building + investing in #ai at @inside Meet Our Fund 5 event! Feel free to check it out 👇
#ai #startups #mof5
youtu.be/JXJqBrGh_5k -
Pieter Abbeel reposted
Philipp Wu
@philippswu
We have released the hardware files and instructions for GELLO 🦾! See the updated website wuphilipp.github.io/gello_site/. Here's a video of me assembling it start to finish in 30 min! GELLO is also at CoRL this week, see 👇 for details
-
Pieter Abbeel reposted
Carlo Sferrazza
@carlo_sferrazza
Training reinforcement learning with tactile feedback in the loop improves its generalization in robotic manipulation… even when the policies do not end up using tactile data! 🧵⬇️
Introducing our recent work at @berkeley_ai w/ @younggyoseo, @haoliuhl, @YoungwoonLee, @pabbeel! -
Pieter Abbeel reposted
Alex Kendall
@alexgkendall
Love this really accessible explanation from @pabbeel about new developments in robot learning. Great interview!
Covariant
@CovariantAI
1/3 BBC Panorama interviewed AI pioneers and industry leaders, including Covariant’s Chief Scientist @pabbeel to unpack this moment for AI and its impact on society.
-
Pieter Abbeel
@pabbeel
Video generation has been one of the main focuses in my lab for several years now. Really exciting to see @ajayj_ and @parasjain taking it to a whole other level post-graduation with Genmo!
Genmo
@genmoai
Introducing Replay v0.2 with a new image-to-video mode.
What’s new?
1) Prompting with an image
2) 3x longer AI video
3) 2.7x higher resolution
Available today. Share your Replay creation below 👇This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Genmo
@genmoai
Introducing Replay v0.2 with a new image-to-video mode.
What’s new?
1) Prompting with an image
2) 3x longer AI video
3) 2.7x higher resolution
Available today. Share your Replay creation below 👇This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
proper
@ProperPrompter
All I can say is "Wow..."
AI video has a new contender with Replay v0.2 by @genmoai
I had the pleasure of doing some testing while it's in early access.
13 short clips showing it's potential: -
Pieter Abbeel reposted
Covariant
@CovariantAI
1/3 BBC Panorama interviewed AI pioneers and industry leaders, including Covariant’s Chief Scientist @pabbeel to unpack this moment for AI and its impact on society.
-
Pieter Abbeel reposted
UC Berkeley IEOR
@BerkeleyIEOR
📢 Faculty Job Alert 📢
Berkeley IEOR is hiring 2 positions:
1️⃣ Assist. Professor in Resilient and Sustainable Supply Chains
2️⃣ Assist. Teaching Professor in Decision Analytics
Learn More: ieor.berkeley.edu/jobs/
Apply & share with your networks! @UCBerkeley @Cal_Engineer -
Pieter Abbeel reposted
Dan Hendrycks
@hendrycks
What can we actually do to reduce risks from AI?
AI researchers Hinton, Bengio, Dawn Song, Pieter Abbeel, and others provide concrete proposals.
managing-ai-risks.com -
Pieter Abbeel
@pabbeel
A lot of the AI safety debate centers around the more extreme views. Excited to be part of a letter that expresses a more nuanced view: managing-ai-risks.com
-
Pieter Abbeel reposted
Sherry Yang
@sherryyangML
Check out Video Language Planning!
VLP uses vision-language models as policy and reward, and UniSim universal-simulator.github.io as dynamics.
Foundation agent models can now conduct search and planning in foundation world models.
Yilun Du
@du_yilun
Introducing Video Language Planning!
video-language-planning.github.io
By planning across the space of generated videos/language, we can synthesize long-horizon video plans and solve much longer horizon tasks than existing baseline (such as RT-2 and PALM-E).
(1/5)This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Boyi Li
@Boyiliee
Can we ask a robot to make boba milk 🧋?
Can we ask a robot to make any drinks based on limited task guidelines? 📒
Can we change our minds and interact with the robot during the execution? 🤖
Yes for all!!
We are excited to announce ITP (Interactive Task Planning with Language Models), an interactive robot framework that accomplishes long-horizon task planning and can easily generalize to new goals or distinct tasks, even during execution.
⭐️ ITP goes through high-level planning to low-level execution, it is easy to be used for other tasks such as Dishwashing.
🌏When tackling various tasks, we can simply substitute the task guidelines and incorporate the relevant low-level execution functions without the need for intricate prompt engineering compared to prior approaches.
Project Page: wuphilipp.github.io/itp_site/
Paper: arxiv.org/abs/2310.10645
Demo: www.youtube.com/watch?v=TrKLuyv26_g
Joint work with the amazing @philippswu @pabbeel @JitendraMalikCV -
Pieter Abbeel reposted
Andy Zeng
@andyzengineer
Large model planners (PaLM-E) generate text 📝 but struggle with physics – can generating videos 🎞️ help?
In "Video Language Planning" we train VLMs + video models to enable robots to imagine (then do) really long multi-step tasks.
Led by the amazing @du_yilun @mengjiao_yang👇
Yilun Du
@du_yilun
Introducing Video Language Planning!
video-language-planning.github.io
By planning across the space of generated videos/language, we can synthesize long-horizon video plans and solve much longer horizon tasks than existing baseline (such as RT-2 and PALM-E).
(1/5)This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Yilun Du
@du_yilun
Introducing Video Language Planning!
video-language-planning.github.io
By planning across the space of generated videos/language, we can synthesize long-horizon video plans and solve much longer horizon tasks than existing baseline (such as RT-2 and PALM-E).
(1/5)This media couldn't be displayed. See it on the original post. -
Pieter Abbeel
@pabbeel
When I started @therobotbrains podcast I had a chat w/ @lexfridman about all things podcasting. I asked: apple, spotify, etc. all combined, how many listens compared to YouTube?
Lex said: ~10x more than YouTube
Just checked for the first time: ~10x it is for most episodes 🤯 -
Pieter Abbeel reposted
Sherry Yang
@sherryyangML
Introducing Universal Simulator (UniSim), an interactive simulator of the real world.
Interactive website: universal-simulator.github.io
Paper: arxiv.org/abs/2310.06114 -
Pieter Abbeel reposted
Hao Liu
@haoliuhl
New paper w/ @matei_zaharia @pabbeel on transformers with large context size.
We propose RingAttention, which allows training sequences that are device count times longer than those of prior state-of-the-arts, without attention approximations or incurring additional overhead. -
Pieter Abbeel reposted
Misha Laskin
@MishaLaskin
Very impressive new work on long context transformers. This particular bit is valuable - going from 4K to 32k context length with a 13B model on just 8 GPUs!
Hao Liu
@haoliuhl
RingAttention lets you scale context length linearly with device count, breaking free from memory constraints. If you could train 4K length on 8 GPU, with RingAttention, you can train at least 32K length with nearly zero overhead
-
Pieter Abbeel reposted
Covariant
@CovariantAI
Hear from @pabbeel: Our robots get things done, no matter how much variation you throw at them.
-
Pieter Abbeel reposted
AIX Ventures
@aixventureshq
Congrats to @aixventureshq portfolio founders @mohak__sharma, @ds3638 at @honeyhiveai on their launch today! Honeyhive is a unified suite of tools and workflows that enable enterprises to scale & automate the process of evaluating, monitoring, & improving LLM apps.
#ai #llmops
Mohak Sharma
@mohak__sharma
I’m excited to announce @HoneyHiveAI is officially coming out of stealth today. Very excited to share what @ds3638 & I have been working on over this past year.
HoneyHive allows developers to test, measure & iteratively improve LLM app performance 📈
honeyhive.ai/blog/announcing-honeyhive -
Pieter Abbeel reposted
Alex Kendall
@alexgkendall
Today we're announcing #gaia1: a 9B parameter world model, trained on 4,700 hours of driving data, able to simulate complex and diverse driving scenes from video, text and action inputs. This model is 480x larger than the preview we shared earlier this year and the results are incredible.
These videos are entirely synthetically generated by @wayve_ai's generative AI, GAIA-1. But there is more here than just generating videos, GAIA is an entire world model. A world model allows us to simulate the future, conditioned on video, text and action inputs, which can be leveraged for making informed decisions when driving.
Why is this game-changing for autonomous driving?
1. Safety. One limitation with AI systems like today's Large Language Models is that they are autoregressive, next-word prediction algorithms, but aren't necessarily aware of the implications of their decisions. A world model allows us to give our AI the capability to be aware of its decisions, by simulating the future, which is important for self-driving safety.
2. Synthetic training data. I believe synthetic training data is the future for AI, because it is safer, cheaper, and infinitely scalable. GAIA-1 unlocks unprecedented realism and diversity of synthetic data for self-driving.
3. Long-tail robustness. One of the biggest challenges for self-driving is long-tail robustness: dealing with the enormous magnitude of edge cases we see on the road. An advantage of generative AI is its incredible ability to recombine experiences in new ways. This is exciting for self-driving as it means we can learn from two edge case scenarios, and combine them to become a corner case. For example, we can experience driving in fog, and experience of jay-walking pedestrians, and GAIA can learn from these experiences to understand how to generate a fog+jay walking scenario.
Check out many more videos in our blog wayve.ai/thinking/scaling-gaia-1/ or further technical details in our paper: arxiv.org/abs/2309.17080
Or come chat with our team who are at the International Conference on Computer Vision (#iccv2023) this week in Paris in Booth 32 @Jamie_Shotton -
Pieter Abbeel reposted
Covariant
@CovariantAI
Automating the manual picking, packing, and sorting bottlenecks in today's fulfillment centers hinges on two key factors: adaptability and reliability.
This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Sharon Zhou
@realSharonZhou
Excited to announce a HUGE secret with @LisaSu: @LaminiAI has been building LLMs on @AMD GPUs *in production* for over the past year!
We’ve made running LLMs on AMD super easy and a highly competitive option through our LLM Superstation, available now at ~10x lower cost than cloud. 👉🏻 www.lamini.ai/blog/lamini-amd-paving-the-road-to-gpu-rich…
Our enterprise customers have already built *thousands* of private LLMs on @LaminiAI LLM Superstations, e.g. @iFit leading at-home fitness with millions of users and @AMD itself:
🚀 Easy & fast: “It was simple to iterate and deploy with a few lines of code and amazingly fast with the AMD Instinct™ hardware.”
⭐️ LLMs are the new IP: “Using a public LLM wasn’t enough: we needed something that we could easily and quickly personalize to our customers’ data and constantly improve on new data, while keeping all of our data private.”
⚙️ Any infrastructure: “We’ve deployed Lamini in our internal Kubernetes cluster with AMD Instinct GPUs, and are using finetuning to create models trained on [our data].”
We had a cameo quote from Joe Spisak at @MetaAI who leads the Llama efforts said: “…Llama 2 is becoming the foundation of some of the most innovative companies.”
🫱🏼🫲🏾 Join Fortune 500 enterprises, and get your own private LLM Superstations—hosted, VPC, or on-premise (just 2 questions): 9gc3kt44b8q.typeform.com/to/mgh0j6Y5
More (technical) details here👉🏻 www.lamini.ai/blog/lamini-amd-paving-the-road-to-gpu-rich… -
Pieter Abbeel
@pabbeel
If you do any AI work in the cloud, so highly recommend checking out @skypilot_org
SkyPilot
@skypilot_org
By adopting SkyPilot, @CovariantAI has transformed their AI development on the cloud, achieving up to 4x faster model shipping speeds while optimizing costs.
Discover their journey to higher AI productivity and elastic scalability.
blog.skypilot.co/covariant/ -
Pieter Abbeel reposted
Philipp Wu
@philippswu
We can also use GELLO for fine-grained manipulation tasks like extracting some floss.
11/n -
Pieter Abbeel reposted
Philipp Wu
@philippswu
🎉Excited to share a fun little hardware project we’ve been working on. GELLO is an intuitive and low cost teleoperation device for robot arms that costs less than $300. We've seen the importance of data quality in imitation learning. Our goal is to make this more accessible
1/n -
Pieter Abbeel reposted
Berkeley AI Research
@berkeley_ai
Recent BAIR alums @ajayj_ and @parasjain are pushing the frontiers of generative diffusion models, with foundational research origins at @UCBerkeley
Genmo
@genmoai
Generative video models are rapidly improving in quality. Meet Replay, a new AI model that can generate stunning videos from text.
Replay v0.1 is designed to create ultrasmooth HD videos with a new interface. Available today for everyone.
What's New?
1. Replay understands plain English prompts without prompt engineering. Try "rugged surfer" or "mermaid".
2. What's a movie without actors? Replay can crisply render close-ups of people and animals.
3. Free and fast generation from our homepage with no waiting list (link in bio).
Behind the scenes, Replay is powered by aligning our new diffusion model for videos with the LLM behind Genmo Chat.
We’re so excited to see what you create! Share your creations below👇 -
Pieter Abbeel reposted
Shyam
@shyammani_
Thrilled to announce that I’m joining AIX Ventures as a Principal in SF! Extremely excited to be investing alongside some of the premier experts in AI including @RichardSocher, @shaunbjohnson, @chrmanning, @pabbeel, @antgoldbloom, and @bianca_caban
-
Pieter Abbeel reposted
Jennifer Chayes
@jenniferchayes
Assistant, Associate and Full Professorships available in AI for Science at UC Berkeley. Join us! aprecruit.berkeley.edu/JPF04118
-
Pieter Abbeel reposted
Aravind Srinivas
@AravSrinivas
Today, we're transforming from a single-player research assistant to a multi-player knowledge platform with the release of Perplexity Collections: pplx.ai/collections. Ask, create, share, and learn together!
-
Pieter Abbeel reposted
Alex Kendall
@alexgkendall
Excellent article which I thought was balanced and insightful 👏, it was awesome to share LINGO-1 with you @strwbilly and @pabbeel 🎉
MIT Technology Review
@techreview
Large language models are the next big thing for robotics, making cars and other robots quicker to train and easier to control... if you trust them. www.technologyreview.com/2023/09/14/1079458/this-driverle…
-
Pieter Abbeel
@pabbeel
Such a great time catching up with Ben Van Roy (web.stanford.edu/~bvr/), who taught me all about function approximation in RL as a student at Stanford. This time teaching me about epistemic neural networks (arxiv.org/abs/2107.08924, w @IanOsband).
-
Pieter Abbeel reposted
Fangchen Liu
@fangchenliu_
We are organizing
@corl_conf 2023 workshop on Reliable and Deployable Learning-Based Robotic Systems with an exciting list of invited speakers, looking towards the future of robot learning systems: corl2023deployable.github.io. Please don't hesitate to submit your work here! -
Pieter Abbeel reposted
Amber Xie
@amberxie_
Excited to share our CoRL 2023 paper: Language-Conditioned Path Planning! ⚡️
Not all collisions are created equal! We address the limitations of traditional path planning by incorporating contact awareness into path planning.
w/ @YoungwoonLee @pabbeel @stepjamUK -
Pieter Abbeel reposted
Aravind Srinivas
@AravSrinivas
For those asking why perplexity is better than bing chat: 28x smaller app, better speed, accuracy, cleaner product, simpler UI and more choices. All done with two or three orders of magnitude fewer resources and marketing, and not having early access to the OpenAI models.
Rob Lennon 🗯 | AI Whisperer
@thatroblennon
This is basically two of my favorite AI tools combined.
If you haven’t used Perplexity lately, give it a whirl. With its super fast web searching, it’s kind of like what Bing Chat never managed to pull off.
Insanely fast and competent for AI knowledge + web search knowledge. -
Pieter Abbeel
@pabbeel
Love this illustration (+the program!)
Matt Bornstein
@BornsteinMatt
[New program] a16z Open Source AI Grants
Hackers & independent devs are massively important to the AI ecosystem.
We're starting a grant funding program so they can continue their work without pressure to generate financial returns.
a16z.com/2023/08/30/supporting-the-open-source-ai-community/ -
Pieter Abbeel reposted
Ideogram
@ideogram_ai
Today, we’re opening Ideogram to everyone on the planet! Sign up at ideogram.ai and have fun!
Ideogram enables you to turn your creative ideas into delightful images, in a matter of seconds. It’s free and has no limits, and it can render text! ideogram.ai/publicly-available -
Pieter Abbeel reposted
Covariant
@CovariantAI
Last week (8/24), our team joined in Covariant’s World Run Day – a 24-hour global relay uniting us across miles and time zones.
The event kicked off with our colleagues in Europe, smoothly handed over to team USA, and was triumphantly concluded by our team in Asia. -
Pieter Abbeel reposted
Index Ventures
@IndexVentures
In the latest episode of @therobotbrains podcast, host @pabbeel chats with Jitendra Malik about captivating AI developmental insights, from sensorimotor learning to language. Plus, don't miss the thought-provoking conversation about the future of AI. www.youtube.com/watch?v=k_Wrd1kI1B0
-
Pieter Abbeel
@pabbeel
Great event coming up next week!
Dawn Song
@dawnsongtweets
Really excited about our Future of Decentralization, AI & Computing Summit, with over 2000 registrations! Keynotes by Shafi Goldwasser of @SimonsInstitute & Michael Jordan of @berkeley_ai & more!
Register & Join Aug 27 (Sunday), UC Berkeley & online: rdi.berkeley.edu/events/decentralizationaisummit -
Pieter Abbeel
@pabbeel
Congratulations @ClementDelangue and entire @huggingface team!
Also, small shameless plug, get to know Clem better here:
www.youtube.com/watch?v=db_ltgsCR54 :)
Yahoo Finance
@YahooFinance
Hugging Face, an AI startup, raised $200 million from investors including $CRM and $NVDA, valuing the company at $4.5 billion. CEO @ClementDelangue speaks with Yahoo Finance to discuss:
This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Covariant
@CovariantAI
Leading the next AI revolution.
@peterxichen joins @twistartups with @Jason to discuss how the Covariant Brain, our universal AI Robotics platform, is transforming logistics and building a foundation of future growth for eCommerce leaders.
Full episode: www.youtube.com/watch?v=shvZDSqD9Co -
Pieter Abbeel reposted
Stephen James
@stepjamUK
Video-language models (VLMs) are often used as a means to solve sparse rewarded tasks. The issue is, they kinda suck giving meaningful rewards 💩.
In this work, we question whether VLMs are perhaps best-suited as a *pretraining* signal for RL.
LAMP💡: arxiv.org/abs/2308.12270
Ademi Adeniji
@AdemiAdeniji
Excited to share LAnguage Reward Modulation for Pretraining Reinforcement Learning!
LAMP💡pretrains a language-conditioned agent without human supervision using VLM rewards and unsupervised reinforcement learning
w/ @amberxie_ @carlo_sferrazza @younggyoseo @stepjamUK @pabbeel -
Pieter Abbeel
@pabbeel
S3 Finale of @therobotbrains: Jitendra Malik on building AI from the ground up: sensorimotor *before* language learning
youtu.be/k_Wrd1kI1B0
Thank you for making the time Jitendra! Thank you to @IndexVentures and @wandb for supporting the podcast production!
Pieter Abbeel
@pabbeel
Don't miss the story on Jitendra's dive bar outing with @rodneyabrooks convincing Jitendra to switch Ph.D. topics 2.5 years in:
www.youtube.com/watch?v=k_Wrd1kI1B0&t=4809s -
Pieter Abbeel reposted
Ademi Adeniji
@AdemiAdeniji
Excited to share LAnguage Reward Modulation for Pretraining Reinforcement Learning!
LAMP💡pretrains a language-conditioned agent without human supervision using VLM rewards and unsupervised reinforcement learning
w/ @amberxie_ @carlo_sferrazza @younggyoseo @stepjamUK @pabbeel -
Pieter Abbeel reposted
Ideogram
@ideogram_ai
We're excited to announce the formation of Ideogram AI today! Our mission is to help people become more creative through Generative AI. ideogram.ai/launch
-
Pieter Abbeel reposted
Joey Gonzalez
@profjoeyg
I started a new blog! Along with @vsreekanti and some of my students at @UCBerkeley, I'll be writing about what I'm seeing in the LLM space across research & industry + what my group is doing. First post is coming later today!
generatingconversation.substack.com -
Pieter Abbeel reposted
Covariant
@CovariantAI
The AI Robotic Revolution is here. And it’s not like what you’ve seen in the movies.
@Forbes sat down with @peterxichen to walk through Covariant’s journey scaling breakthrough AI Robotics to pioneer a pragmatic revolution that is redefining how we work. www.forbes.com/sites/hessiejones/2023/08/15/has-ai-roboti… -
Pieter Abbeel
@pabbeel
Hard to imagine a better way for me to flash right back into @OpenAI early days 2016, inspiring clarity of thinking and communication by @ilyasut
www.youtube.com/live/AKMuA_TVz3A -
Pieter Abbeel
@pabbeel
check out @BBCPanorama covering our @berkeley_ai robot dog learning to get up and walk from its own trial and error! As well as many other topics in AI, with the episode also featuring @ylecun and Yoshua Bengio
BBC Panorama
@BBCPanorama
This robot dog is learning to walk by trial and error - rather than have its movements decided by its creators
This technology of “re-enforcement learning” is also behind the AI chatbot ChatGPT
On BBC iPlayer now
www.bbc.co.uk/programmes/m001ph7q?at_bbc_team=editorialThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Covariant
@CovariantAI
Tune in as @pabbeel and other leading experts explore the dynamic between modern AI and our world
BBC Panorama
@BBCPanorama
Machines have always needed humans to tell them what to do
But the artificial intelligence revolution is changing that
Beyond Human: Artificial Intelligence and Us is on BBC iPlayer now and BBC One at 8pm
www.bbc.co.uk/programmes/m001ph7q -
Pieter Abbeel
@pabbeel
My conversation with Seb Boyer from Farmwise on AI robotic weeding, future of farming, and a history of tech in farming!
youtu.be/Iad_LZxZqbc
Thank you for making the time Seb! A special thank you to @IndexVentures and @wandb for supporting the podcast production! -
Pieter Abbeel reposted
Covariant
@CovariantAI
No more hardcoded solutions, just adaptive AI that picks and places your diverse and ever-changing item mix with ease.
Watch how the Covariant Brain’s high-precision visualization revolutionizes apparel handling. -
Pieter Abbeel
@pabbeel
This is amazing, AI powered mechanical precision weeding, no need for chemicals! Farmwise founding CEO Seb Boyer explains this and more in the next episode of @therobotbrains!
Agri Machines
@AgTech_360
Automated Weeding System || Made By FarmWise USA || #shorts #autonomous #weeding
-
Pieter Abbeel reposted
Covariant
@CovariantAI
Following the journey from a PhD student in @pabbeel’s lab to co-founding OpenAI and leading the development of ChatGPT — hear from @johnschulman2 on @therobotbrains on how #chatgpt is built.
This media couldn't be displayed. See it on the original post. -
Pieter Abbeel
@pabbeel
My conversation with @johnschulman2 of OpenAI on ChatGPT: invention, capabilities and limitations
youtu.be/nM_3d37lmcM
Pieter Abbeel
@pabbeel
Thanks so much for making the time John, and also a special thank you to @IndexVentures and @wandb for supporting the production of the podcast!
-
Pieter Abbeel reposted
Jessy Lin
@realJessyLin
How can agents understand the world from diverse language? 🌎
Excited to introduce Dynalang, an agent that learns to understand language by 𝙢𝙖𝙠𝙞𝙣𝙜 𝙥𝙧𝙚𝙙𝙞𝙘𝙩𝙞𝙤𝙣𝙨 𝙖𝙗𝙤𝙪𝙩 𝙩𝙝𝙚 𝙛𝙪𝙩𝙪𝙧𝙚 with a multimodal world model! -
Pieter Abbeel reposted
Covariant
@CovariantAI
Enabling robots with human-like intelligence.
Tune into our customer @OttoGroup_Com's latest podcast on Spotify with @peterxichen where he discusses the essence of robotic intelligence: open.spotify.com/episode/0MEwIVfwostD7cmkKTLaKZ -
Pieter Abbeel reposted
Anirudha Majumdar
@Majumdar_Ani
With seemingly endless progress in AI, I decided over the past few months to take a deep dive into the state of robotics 🤖, emerging trends, and research challenges. What do actual field-deployed robotic systems look like now? Will LLMs solve robotics?
irom-lab.princeton.edu/wp-content/uploads/2023/08/Robotic…
🧵 -
Pieter Abbeel reposted
Covariant
@CovariantAI
From bins to bags, our robot ensures your groceries are in safe hands – or grippers.
-
Pieter Abbeel reposted
Sam Gershman
@gershbrain
I plan to hire a postdoc next summer to work on machine learning projects at the interface with cognitive science and neuroscience. Part of a collaboration with @pulkitology, @pabbeel, @AudeOliva, and @FieteGroup. Get in touch with me if you're interested.
-
Pieter Abbeel
@pabbeel
Cow burps and farts contribute rather significantly to carbon emissions. Turns out AI can help!
youtu.be/p0nX5QtLW_c
Thank you so much @AltshulerYaniv for telling the story of @MethaAi making such a dent in this space!Post not available here, view on X
Pieter Abbeel
@pabbeel
On other platforms:
Apple: podcasts.apple.com/us/podcast/yaniv-altshuler-reducing-co…
Spotify: open.spotify.com/episode/4DAKLPJ9pmvNqjKSGjvG1g
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/yaniv-altshuler…
Pieter Abbeel
@pabbeel
Also a special thank you to @IndexVentures and @wandb for supporting the production of the podcast!
-
Pieter Abbeel reposted
-
Pieter Abbeel reposted
Paul Graham
@paulg
The essay I just published called "How to Do Great Work" (paulgraham.com/greatwork.html) grew out of a single paragraph in another essay I was writing. It seemed such an important topic that I cut it out and made it into its own essay.
-
Pieter Abbeel reposted
Index Ventures
@IndexVentures
On this week’s @therobotbrains podcast episode, @pabbeel talks with @NASA astronaut @Astro_Woody live from the ISS. 🛰️ Discover the journey to the ISS, life in weightlessness, impactful experiments, and the potential for low earth orbit factories. www.youtube.com/watch?v=ItopcoP8ST0
-
Pieter Abbeel reposted
Berkeley AI Research
@berkeley_ai
@therobotbrains podcast reaches space! Check out this phenomenal episode with BAIR graduate @Astro_Woody, straight from the International Space Station
Pieter Abbeel
@pabbeel
Had the amazing opportunity to talk with @NASA Astronaut @Astro_Woody, who piloted the @SpaceX Crew-6 rocket, and is currently at the ISS:
youtu.be/ItopcoP8ST0 -
Pieter Abbeel
@pabbeel
Had the amazing opportunity to talk with @NASA Astronaut @Astro_Woody, who piloted the @SpaceX Crew-6 rocket, and is currently at the ISS:
youtu.be/ItopcoP8ST0Post not available here, view on X
Pieter Abbeel
@pabbeel
-- Impact of work at ISS on Humanity
www.youtube.com/watch?v=ItopcoP8ST0&t=340s
-- Scientific experiments at ISS
www.youtube.com/watch?v=ItopcoP8ST0&t=478s
-- Low earth orbit factories
www.youtube.com/watch?v=ItopcoP8ST0&t=719s
-- Moon and Mars
www.youtube.com/watch?v=ItopcoP8ST0&t=880s
Pieter Abbeel
@pabbeel
The Robot Brains Podcast on other platforms:
Apple: podcasts.apple.com/us/podcast/woody-hoburg-nasa-astronaut…
Spotify: open.spotify.com/episode/0YdAeMelDkQDNroNG9d1UV
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/woody-hoburg-na…
Pieter Abbeel
@pabbeel
Thanks so much @Astro_Woody for making the time! Also a special thank you to @IndexVentures
and @wandb for supporting the production of the podcast! -
Pieter Abbeel reposted
Xingyu Lin
@Xingyu2017
Excited to share my first paper at UC Berkeley! We identify key bottlenecks in learning from a pre-trained visual representation and show generalization to novel objects from only three instances.🧵⬇️
Website: xingyu-lin.github.io/spawnnet/ -
Pieter Abbeel
@pabbeel
Is it crazy to not only build the AI for self-driving, but ALSO the re-imagined car and ALSO the ride-sharing experience?
Given how long/hard the AI has proven anyway, maybe not!
Here is my conversation with @zoox co-founder/CTO Jesse Levinson:
youtu.be/3_g14RAyOzEPost not available here, view on X
Pieter Abbeel
@pabbeel
Our podcast on other platforms:
Apple: podcasts.apple.com/us/podcast/jesse-levinson-of-zoox-rein…
Spotify: open.spotify.com/episode/02edJ5fSWoJjoAplzBJUj9
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/jesse-levinson-…
Pieter Abbeel
@pabbeel
Thanks so much Jesse Levinson (@zoox)
for making the time! Also a special thank you to @IndexVentures and @wandb for supporting the production of the podcast! -
Pieter Abbeel
@pabbeel
Very excited to share my conversation with Noam Brown @polynoamial on building AI to solve poker and diplomacy, and how the same ideas will matter for LLMs.
youtu.be/ceCg90Q9N6YPost not available here, view on X
Pieter Abbeel
@pabbeel
Our podcast on other platforms:
Apple: podcasts.apple.com/us/podcast/noam-brown-from-open-ai-on-…
Spotify: open.spotify.com/episode/0Flbwb02e2boEHbR7mxu7r
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/noam-brown
Pieter Abbeel
@pabbeel
Thanks so much Noam Brown @polynoamial
for making the time! Also a special thank you to @IndexVentures and @wandb for supporting the production of the podcast! -
Pieter Abbeel reposted
Simon Arkell
@sarkell
Delighted to announce our new company, Synthetica Bio, our new venture capital funding from Virtue , AIX Ventures , Page One Ventures and others, and our great team that is dedicated to helping clients in the biopharma industry.
www.linkedin.com/posts/simonarkell_synthetica-bio-launche… -
Pieter Abbeel reposted
Joey Hejna
@JoeyHejna
Our work from AAAI 2023 is now on arxiv (very overdue)! The idea is really simple -- predicting language instructions during training improves generalization at test-time, particularly for combinatorial tasks.
-
Pieter Abbeel reposted
Lambda
@LambdaAPI
Lambda Co-founder and CEO @stephenbalaban sat down with @pabbeel on the latest @therobotbrains episode to talk about the current state of the GPU cloud market and our journey to building the largest and most cost-effective GPU cloud in the world: www.youtube.com/watch?v=hcKSwiNk-CA
-
Pieter Abbeel
@pabbeel
Excited to share my conversation with @stephenbalaban from @LambdaAPI, who are on a mission to build the largest and most cost-effective AI cloud
youtu.be/hcKSwiNk-CAPost not available here, view on X
Pieter Abbeel
@pabbeel
Our podcast on other platforms:
Apple: podcasts.apple.com/us/podcast/stephen-balaban-building-th…
Spotify: open.spotify.com/episode/5BBblc0iPA7JxA2YR0Ol5l
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/stephen-balaban…
Pieter Abbeel
@pabbeel
Thanks so much @stephenbalaban for making the time!
Also a special thank you to @IndexVentures
and @wandb for supporting the production of the podcast! -
Pieter Abbeel reposted
stephen balaban
@stephenbalaban
Sat down with Pieter Abbeel @pabbeel to talk about the current state of the GPU cloud market, Lambda's founding history, and the deep learning powered future of gaming. @therobotbrains podcast is great.
www.youtube.com/watch?v=hcKSwiNk-CA -
Pieter Abbeel
@pabbeel
Our @therobotbrains guest @ralucaadapopa is organizing the first Confidential Computing Summit! Details in original tweet. Link to our conversation, here:
youtu.be/YqCEk0eyVy8
Raluca Ada Popa
@ralucaadapopa
Less than 2 weeks until the Confidential Computing Summit, the main in-person CC event! We have an impressive lineup of speakers on generative AI privacy, confidential data collaboration & data clean rooms etc.
REGISTER before tickets run out: confidentialcomputingsummit.com -
Pieter Abbeel
@pabbeel
video prediction models keep getting better
Jamie Shotton
@Jamie_Shotton
I’m incredibly excited to share a sneak peak of @wayve_ai’s GAIA-1, a major step forward in action and language-conditioned world models for #autonomousdriving. Here’s a glimpse of what GAIA-1 can do - more in the 🧵. (These are not real videos!) But what's a world model?
-
Pieter Abbeel
@pabbeel
Really enjoyed discussing the role of foundation models and generative AI for robotics with Brian Heater from @TechCrunch
TechCrunch
@TechCrunch
Pieter Abbeel and Ken Goldberg on generative AI applications tcrn.ch/3Pl3cXv by @bheater
-
Pieter Abbeel reposted
Aravind Srinivas
@AravSrinivas
We're excited to announce Perplexity Pro:
1. Practically unlimited Copilot searches (300 + a day)
2. GPT-4 for all search queries
3. GPT-4 for text generation as a focused use case
4. Access to new upgrades which will be rolled out continuously over the coming weeks
5. Discord server with fast support from the team
All at 20$/mo (or 200$/yr)!
At this price point, you get everything you need from a Generative AI subscription service for text-based knowledge-work (research, writing, coding, travel, shopping, trivia) and more to come!
Perplexity
@perplexity_ai
⚡️Supercharge your search with Perplexity Pro! Unlock nearly unlimited access to our most advanced search Copilot, the latest AI models, Pro support, and more - all for just $20/month or $200/year. 🚀
Get started at perplexity.ai/pro -
Pieter Abbeel
@pabbeel
Excited to share my conversation with @rocky_duan from OpenAI to Covariant: Building Foundation Models for Robotic Automation
youtu.be/QhDLHWR04RkPost not available here, view on X
Pieter Abbeel
@pabbeel
Our podcast on other platforms:
Apple: podcasts.apple.com/us/podcast/rocky-duan-from-openai-to-c…
Spotify: open.spotify.com/episode/4uIzVcEXqmfRw8Snr7JqnK
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/rocky-duan-cova…
Pieter Abbeel
@pabbeel
Thanks so much @rocky_duan for making the time, and also a special thank you to @IndexVentures and @wandb for supporting the production of the podcast!
-
Pieter Abbeel reposted
Covariant
@CovariantAI
At just 21, Rocky Duan became one of @OpenAI's early hires, pushing boundaries on #reinforcementlearning, and imitation learning.
At 23, he went on co-found Covariant bringing breakthrough ML concepts to robotic automation for #warehousing, where he continues to serve as CTO. -
Pieter Abbeel reposted
Sherry Yang
@sherryyangML
As video foundation models reach billions of parameters, how to adapt them to task-specific settings (e.g., animation, robotics) without access to the model weights becomes a pressing issue.
We introduce Video Adapter:
arxiv.org/abs/2306.01872
video-adapter.github.io -
Pieter Abbeel reposted
Alejandro Escontrela
@alescontrela
If you want to train agents from raw unlabeled videos, the code for Video Prediction Rewards (VIPER) 🐍 is now publicly available.
VIPER trains agents from pixels, and supports continuous/discrete action spaces.
github.com/alescontrela/viper_rl -
Pieter Abbeel reposted
Hao Liu
@haoliuhl
1/ Excited to share our new paper with @pabbeel on long context models! 📚✍️ Check it out here: arxiv.org/abs/2305.19370
Training 7B models with over 130K or 13B models with over 64K context windows on just 8 A100 GPUs! 😮🖥️
Curious how we did it? -
Pieter Abbeel reposted
Berkeley AI Research
@berkeley_ai
New BAIR Blog post on combining LLMs like GPT-4 with text-to-image diffusion models. bair.berkeley.edu/blog/2023/05/23/lmd/
-
Pieter Abbeel
@pabbeel
Wonderful conversation with Kiva Systems (-> AMZN) founder, Verity founder, ETHZ Prof @raffdandrea, spanning drones, warehouse automation, actuated wingsuits, art, and even Maxwell's Demon.
www.youtube.com/watch?v=DgwhJnkmxZwPost not available here, view on X
Pieter Abbeel
@pabbeel
On other platforms:
Apple: podcasts.apple.com/us/podcast/raffaello-dandrea-of-verity…
Spotify: open.spotify.com/episode/27BUFump2Dj25pQYlXWn5A
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/raffaello-dandrea
Pieter Abbeel
@pabbeel
Thanks so much @raffdandrea for making the time, and also a special thank you to @IndexVentures and @wandb for supporting the production of the podcast!
-
Pieter Abbeel reposted
The Full Stack
@full_stack_dl
🆕 LLM Bootcamp videos are now available!
Check out our awesome invited speakers:
🏋🏻 @truerezashabani walks us through training LLMs at @Replit
🕵🏽 @hwchase17 talks about building agents with @langchain
🔥 @npew talks about the path to @OpenAI ChatGPT -
Pieter Abbeel reposted
Covariant
@CovariantAI
Don’t take it from us.
Read how @OttoGroup_Com is deploying 120 Covariant AI-powered #robots to further solidify their place as an #ecommerce leader.
www.ottogroup.com/en/stories/story/artificial-intelligenc… -
Pieter Abbeel reposted
Kimin
@kimin_le2
❓ What is an effective approach for fine-tuning pre-trained t2i diffusion models using a reward function?
💡 I'm excited to share "DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models" co-led by @yingfan_bot
Website: sites.google.com/view/dpok-t2i-diffusion/home
🧵 1/N -
Pieter Abbeel reposted
Mohak Sharma
@mohak__sharma
@honeyhiveai is growing! 🌱
We're hiring exceptional founding engineers to help developers safely and reliably build with LLMs in production. Great opportunity if you're looking to get deeply involved in LLM research & tooling!
Apply below 👇 careers.honeyhive.ai -
Pieter Abbeel reposted
Alejandro Escontrela
@alescontrela
Today we're releasing Video Prediction Rewards (VIPER 🐍), a simple yet powerful method for extracting rewards from video prediction models!
VIPER learns reward functions from raw videos, and generalizes to entirely new domains for which no training data is available
🧵 thread -
Pieter Abbeel reposted
Cohere
@cohere
1/ 📢The future of NLP education is here!
We are proud to announce the official launch of LLM University (LLMU) 🏫
🧵
txt.cohere.com/llm-university -
Pieter Abbeel reposted
Perplexity
@perplexity_ai
The next iteration of Perplexity has arrived: Copilot, your interactive AI search companion. 🚀🤖 Perplexity Copilot guides your search experience with interactive inputs, leading you to a rich, personalized answer, powered by GPT-4. Try it for free at perplexity.ai
-
Pieter Abbeel reposted
Covariant
@CovariantAI
Fashion is fast, your fulfillment needs to be faster.
A @McKinsey retail report says 90% of customers expect a two- to three-day delivery window.
Our latest blog breaks down why apparel retailers are leading the industry in material handling investments: covariant.ai/resources/report-apparel-retailers-lead-the-… -
Pieter Abbeel
@pabbeel
Wonderful chat with @YejinChoinka on instilling AI with common sense and morality!
youtu.be/MKyTVPCVoWgPost not available here, view on X
Pieter Abbeel
@pabbeel
-- The power of high quality data
www.youtube.com/watch?v=MKyTVPCVoWg&t=1635s
-- Common sense
www.youtube.com/watch?v=MKyTVPCVoWg&t=1748s
-- Language as the best medium for reasoning
www.youtube.com/watch?v=MKyTVPCVoWg&t=1939s
Pieter Abbeel
@pabbeel
-- Can machines learn morality
www.youtube.com/watch?v=MKyTVPCVoWg&t=2140s
-- Any benefits to smaller models
www.youtube.com/watch?v=MKyTVPCVoWg&t=2409s
-- Syntax, semantics, pragmatics
www.youtube.com/watch?v=MKyTVPCVoWg&t=2493s
Pieter Abbeel
@pabbeel
On other platforms:
Apple: podcasts.apple.com/us/podcast/yejin-choi-teaching-ai-comm…
Spotify: open.spotify.com/episode/1e3EtL6K2HZTJktHzESfCO
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/yejin-choi
Pieter Abbeel
@pabbeel
Thanks so much @YejinChoinka for making the time, and also a special thank you to @IndexVentures and @wandb for supporting the production of the podcast!
-
Pieter Abbeel reposted
The Full Stack
@full_stack_dl
🥞🦜 Full Stack LLM Bootcamp 🦜🥞
tl;dr We're releasing our lectures on building LLM-powered apps, for FREE.
🚀 Launch an LLM App in One Hour
✨ Prompt Engineering
🗿 LLM Foundations
🔨 Augmented LLMs
🤷 UX for LUIs
🏎️ LLMOps
🔮 What's Next?
👷 Project Walkthrough
Learn more: -
Pieter Abbeel
@pabbeel
Thank you @geoffreyhinton for diving deeper into the major potential risks you see with AI, and also reminding us of the tremendous potential for AI to improve our lives.
youtu.be/rLG68k2blOcPost not available here, view on X
Pieter Abbeel
@pabbeel
Our podcast on other platforms:
Apple: podcasts.apple.com/us/podcast/geoff-hinton-returns-the-go…
Spotify: open.spotify.com/episode/5fqYMFp8JEoJmCwkRxJJee
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/geoff-hinton-th…
Pieter Abbeel
@pabbeel
Thanks so much @geoffreyhinton for making the time, and also a special thank you to @IndexVentures and @wandb for supporting the production of the podcast!
-
Pieter Abbeel reposted
Lukas Biewald
@l2k
Really enjoyed being on @pabbeel's @therobotbrains podcast. We talk about everything from hyper parameter optimization to parenting.
www.youtube.com/watch?v=tG0HGKNnp-A -
Pieter Abbeel reposted
CoreWeave Developers
@coreweavedevs
New episode of @therobotbrains with W&B Cofounder and CEO @l2k 🤖 🧠 🎧
Host @pabbeel and Lukas discuss solving the pain-points of AI researchers and practitioners 🫵
www.youtube.com/watch?v=tG0HGKNnp-A -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
This week's guest @l2k shared some resources for getting started in AI:
@twominutepapers: www.youtube.com/@twominutepapers
@wandb ML courses: www.wandb.courses/pages/w-b-c...
@3blue1brown's: www.youtube.com/@3blue1brown/playlists -
Pieter Abbeel reposted
Aravind Srinivas
@AravSrinivas
We're excited to announce our collaboration with @Wolfram_Alpha to give all @perplexity_ai users a chance to experience some Wolfram magic for computational and knowledge queries, whether it be helping you with integrals and derivatives or asking for accurate and reliable information about cities and planets! Start asking at perplexity.ai
Perplexity
@perplexity_ai
Unlock the power of computational knowledge plus large language models with Perplexity AI + @Wolfram_Alpha 💪 Quickly and easily solve complex problems with accurate, detailed results. 🚀 Try it out today for free at perplexity.ai.
-
Pieter Abbeel
@pabbeel
"the old dude that created the AI" (aka the amazing @geoffreyhinton!) will be back on @therobotbrains podcast next week, talking freely! cc @SnoopDogg
-
Pieter Abbeel
@pabbeel
New episode out with @l2k founder/CEO of @wandb on building products that solve the pain points of AI researchers and practitioners!
www.youtube.com/watch?v=tG0HGKNnp-APost not available here, view on X
Pieter Abbeel
@pabbeel
-- From Stanford to founding Crowdflower / Figure Eight
www.youtube.com/watch?v=tG0HGKNnp-A&t=1742s
-- Parenting
www.youtube.com/watch?v=tG0HGKNnp-A&t=2552s
-- Resources to get started in AI
www.youtube.com/watch?v=tG0HGKNnp-A&t=2830s
-- Future of AI
www.youtube.com/watch?v=tG0HGKNnp-A&t=3060s
Pieter Abbeel
@pabbeel
Our podcast on other platforms:
Apple: podcasts.apple.com/us/podcast/lukas-biewald-of-weights-an…
Spotify: open.spotify.com/episode/4mWfPecFyJfSOWyBuJIbu0
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/lukas-biewald-o…
Pieter Abbeel
@pabbeel
Thanks so much @l2k for making the time, and also a special thank you to @IndexVentures and @wandb for supporting the production of the podcast!
Pieter Abbeel
@pabbeel
BONUS Feature: Lukas Biewald (@l2k) gives me a quick screenshare overview of @wandb:
www.youtube.com/watch?v=R9SLjTxpdW4 -
Pieter Abbeel reposted
Hao Liu
@haoliuhl
As a part of our effort to replicate LLaMA in an open-source manner, we are pleased to announce the release of preview of the 7B OpenLLaMA model that has been trained with 200 billion tokens on the RedPajama dataset.
github.com/openlm-research/open_llama -
Pieter Abbeel
@pabbeel
Check out our recent work on MTM, a new self-supervised paradigm for RL.
MTM aims to reconstruct randomly dropped out elements in trajectories, analogous to BERT/MAE for NLP/CV. Resulting model can be used for state representations, world models, offline RL, and more. See 🧵:
Philipp Wu
@philippswu
Introducing Masked Trajectory Modeling (MTM), a new general-purpose framework for sequential decision making. A single transformer trained with MTM can exhibit multiple capabilities by simply choosing different masking patterns at inference time. Accepted at ICML 2023. 🧵👇
-
Pieter Abbeel
@pabbeel
New @therobotbrains episode with guest @NandoDF from @DeepMind! We discuss Generalizable AI and how to ensure AI benefits everyone.
youtu.be/dKvYRgOifFAPost not available here, view on X
Pieter Abbeel
@pabbeel
Our podcast on other platforms:
Apple: podcasts.apple.com/us/podcast/nando-de-freitas-of-deepmin…
Spotify: open.spotify.com/episode/5xzzdYoo1DpgaJb6EQWegc
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/nando-de-freitas
Pieter Abbeel
@pabbeel
Thanks so much @NandoDF for making the time, and also a special thank you to @IndexVentures and
@wandb for supporting the production of the podcast!Post not available here, view on X
Pieter Abbeel
@pabbeel
Thanks so much @NandoDF for making the time, and also a special thank you to @IndexVentures and @wandb for supporting the production of the podcast!
-
Post not available here, view on X
Pieter Abbeel
@pabbeel
Direct links to some of my favorite parts:
- making sure AI benefits the whole world
www.youtube.com/watch?v=dKvYRgOifFA&t=284s
- AlphaCode
www.youtube.com/watch?v=dKvYRgOifFA&t=1192s
- Gato: a Generalist Agent
www.youtube.com/watch?v=dKvYRgOifFA&t=1535s
Pieter Abbeel
@pabbeel
- using videos to train AI agents
www.youtube.com/watch?v=dKvYRgOifFA&t=2271s
- learning to learn
www.youtube.com/watch?v=dKvYRgOifFA&t=2914s
- supplementing LLMs with human-like methods of communication to solve problems
www.youtube.com/watch?v=dKvYRgOifFA&t=3282s -
Pieter Abbeel reposted
Ge Yang
@EpisodeYang
@johnschulman2 's talk at BAIR today on ChatGPT titled RL and Truthfulness --- Towards TruthGPT.
What a time!
www.youtube.com/live/hhiLw5Q_UFg?t=911 -
Pieter Abbeel reposted
Eric Jang
@ericjang11
Thanks @johnschulman2 for the amazing talk! "Open Problems I, II, III" slides paint a really compelling sequence of where these models are headed. Now we just need to predict the next slide, "Open Problems IV"...
www.youtube.com/live/hhiLw5Q_UFg via @YouTube -
Pieter Abbeel reposted
Aravind Srinivas
@AravSrinivas
Everyone's been asking us for the @perplexity_ai Android app. The time has come to get on the waitlist. Coming soon to the Play Store! perplexity.ai/android
-
Pieter Abbeel reposted
Ken Goldberg
@Ken_Goldberg
Today at 5:10pm PST: Talk by #chatgpt lead engineer and @UCBerkeley alum John Schulman: Reinforcement Learning from Human Feedback: Progress and Challenges. Berkeley AI Research Lab @berkeley_ai @Berkeley_EECS Livestreamed: www.youtube.com/c/UCBerkeleyEECSEvents/live
-
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
@RichardSocher of @YouSearchEngine joins @pabbeel to discuss the future of search, the early days of Deep Learning for NLPs, achieving AGI, and the founding @aixventureshq.
Watch here ⬇️ www.youtube.com/watch?v=v2UDEHvtONQ -
Pieter Abbeel
@pabbeel
As you might have noticed, we recently raised another big round. AI Robotic automation for warehousing/logistics is happening now, and a lot of exciting work ahead. Looking for passionate engineers to join us at @CovariantAI!
Open positions: covariant.ai/careers/ -
Pieter Abbeel
@pabbeel
New @therobotbrains episode with guest
@RichardSocher! We discuss LLMs, Search, you.com, AGI, Metamind, AIX Ventures.
youtu.be/v2UDEHvtONQ
📷Post not available here, view on X
Pieter Abbeel
@pabbeel
Our podcast on other platforms:
Apple: podcasts.apple.com/us/podcast/richard-socher-of-you-com-t…
Spotify: open.spotify.com/episode/0X9tZNsy7uoJYm5m9oGmtt
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/richard-socher-…
Pieter Abbeel
@pabbeel
Thanks so much @RichardSocher for making the time, and also a special thank you to @IndexVentures
and @wandb for supporting the production of the podcast! -
Pieter Abbeel reposted
-
Pieter Abbeel reposted
Aravind Srinivas
@AravSrinivas
We are excited to transition perplexity.ai to an even more mature consumer platform for knowledge and productivity, supporting users to sign up, maintain a history of past threads, curate, and share them!
Perplexity
@perplexity_ai
We are excited to launch the next version of perplexity.ai! Introducing login, threads, focus search, improved formatting, and more. 🎉
Login now to start collecting your own library of threads. Keep them to yourself, or share your latest discovery. You decide. -
Pieter Abbeel reposted
Bianca Caban
@bianca_caban
Interesting insight from @aixventureshq investment partner @antgoldbloom on @i3invest 's pod on how the capabilities of #ml are outpacing company formation and new applications. There's so much yet to come in terms of what founders will build with models like ChatGPT 🙌💡
[i3]
@i3invest
In Episode 82 of the [i3] Podcast, we speak with Anthony Goldbloom and Christopher Manning of @aixventureshq about #chatgpt, #machinelearning and #venturecapital. Full podcast next week:
i3-invest.com/podcasts/ #podcast #preview -
Pieter Abbeel
@pabbeel
New @therobotbrains episode with guest @ralucaadapopa! We discuss Data Privacy and AI, Cloud Enclaves, Confidential and Encrypted Computing, Differential Privacy, PreVeil, Opaque Systems
youtu.be/YqCEk0eyVy8Post not available here, view on X
Pieter Abbeel
@pabbeel
Links on other platforms:
Apple: podcasts.apple.com/us/podcast/raluca-ada-popa-how-to-trai…
Spotify: spoti.fi/3zTN3iG
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/raluca-ada-popa
Pieter Abbeel
@pabbeel
Thanks so much @ralucaadapopa for making the time, and also a special thank you to @IndexVentures
and @wandb for supporting the production of the podcast! -
Pieter Abbeel reposted
anton 🇺🇸
@atroyn
@trychroma growth since we launched on feb 14 has been insane.
contributions from our community are helping us build the best, easiest to use memory layer for a.i-native applications.
here are some recent contributions -
Pieter Abbeel reposted
Covariant
@CovariantAI
We are breaking new ground together. Happy National Robotics Week.
#roboweek #nationalroboticsweek -
Pieter Abbeel reposted
Index Ventures
@IndexVentures
On this week’s episode of @therobotbrains podcast, @aidangomezzz from @CohereAI and @pabbeel from @CovariantAI dive into the 🌎 of adoption and use cases, and touch on Cohere for AI, a non-profit organization, which aims to make AI accessible to all.
www.youtube.com/watch?v=zBK2CPka5jo -
Pieter Abbeel reposted
AIX Ventures
@aixventureshq
Congrats to @aixventureshq portco founders @jeffreryhuber and @atroyn of @trychroma on their $18M raise! We're proud to have backed them early on and are excited for their awesome momentum on their AI-native open-source embeddings database.
#venturecapital #ai #startups
Jeff Huber
@jeffreyhuber
I'm thrilled to announce Chroma's $18M seed round led by Quiet Capital
We are excited to continue to invest deeply in the project and make it a ubiquitous open source standard
www.trychroma.com/blog/seed -
Pieter Abbeel reposted
Stephanie Palazzolo
@steph_palazzolo
Looking to learn more about AI? Now, it can be as easy as popping on one of these 11 podcasts during your morning commute.
Featuring pods like @saranormous and @eladgil's @NoPriorsPod, @pabbeel's @therobotbrains, and more:
www.businessinsider.com/top-artificial-intelligence-machi… -
Pieter Abbeel reposted
Aravind Srinivas
@AravSrinivas
The power of building a general purpose answer engine rather than targeting a specific vertical: the same app that can help you learn more about nuclear fusion can also help you plan a summer vacation, without actually compromising on the search experience. Start asking at
Perplexity
@perplexity_ai
Travel planning made easy with Perplexity's iPhone app! Ask questions and get relevant answers without the hassle of sifting through the internet. Find the best time to visit the destination you've been thinking about, discover related questions, create a packing list, and find flight deals – all on your own terms. Save time and effort while we help you plan your perfect trip. Download the app now and start asking! www.perplexity.ai/download-iphone
-
Pieter Abbeel reposted
JustRecap
@JustRecapIt
Heya, a recap of the video is ready!
reccap.it/recaps/s3-e4-aidan-gomez-joins-host-pieter-abbe… -
Pieter Abbeel reposted
Suchi Saria
@suchisaria
I’ve been using the @perplexity_ai app on my phone for search and it’s been a great experience. Rarely go to chrome now.
-
Pieter Abbeel reposted
Aidan Gomez
@aidangomez
Really enjoyed this conversation with Pieter. We covered Cohere’s thesis and my view on the tech roadmap for the field.
Pieter Abbeel
@pabbeel
New episode, with @aidangomezzz from @CohereAI!
We discuss the Transformers paper, Large Language Models, Command and Instruct Models, Action beyond Text Completion. @therobotbrains
youtu.be/zBK2CPka5jo -
Pieter Abbeel
@pabbeel
New episode, with @aidangomezzz from @CohereAI!
We discuss the Transformers paper, Large Language Models, Command and Instruct Models, Action beyond Text Completion. @therobotbrains
youtu.be/zBK2CPka5joPost not available here, view on X
Pieter Abbeel
@pabbeel
Links on other platforms:
Apple: podcasts.apple.com/us/podcast/aidan-gomez-of-cohere-build…
Spotify: open.spotify.com/episode/4IcxiUslgMlsF3pO4nQi47
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/aidan-gomez-of-…
Pieter Abbeel
@pabbeel
Thanks so much @aidangomezzz for making the time, and a special thank you also to @IndexVentures and @wandb for supporting the production of the podcast!
-
Pieter Abbeel reposted
AIX Ventures
@aixventureshq
We're thrilled to back AIX Ventures Investment Partner @pabbeel and the team @CovariantAI as they continue to pioneer AI-powered robotic automation in the warehouse!
#ai #robotics #supplychain
Covariant
@CovariantAI
We are excited to announce our funding of $75M to meet growing demand for increased warehouse efficiency and resilience through AI-powered robotic automation.
techcrunch.com/2023/04/04/covariants-robotic-picking-at-n… -
Pieter Abbeel reposted
Amplify Partners
@AmplifyPartners
Congratulations to the @CovariantAI team on another round of funding to further the deployment of Covariant’s system to retailers and logistics firms! CEO @peterxichen shares more with @bheater:
techcrunch.com/2023/04/04/covariants-robotic-picking-at-n…
Covariant
@CovariantAI
We are excited to announce our funding of $75M to meet growing demand for increased warehouse efficiency and resilience through AI-powered robotic automation.
techcrunch.com/2023/04/04/covariants-robotic-picking-at-n… -
Pieter Abbeel reposted
Lior Alexander
@LiorOnAI
Berkley just released Koala-13B!
The open-source chatbot was trained by fine-tuning LLaMA on web dialogue! 50% of responses are similar to ChatGPT.
The paper also suggests that training with high-quality datasets can compensate for smaller models, possibly matching larger ones -
Pieter Abbeel reposted
Lior Alexander
@LiorOnAI
Blog:
bair.berkeley.edu/blog/2023/04/03/koala/
Demo:
chat.lmsys.org?model=koala-13b
Work led by @haoliuhl, @Eric_Wallace_, @ArnavGudibande, Xinyang Geng, @pabbeel @svlevine , @dawnsongtweets, @berkeley_ai -
Pieter Abbeel reposted
Mike Volpi
@mavolpi
Nice work by @peterxichen @pabbeel to load up on cash for the long haul. The move to warehouse robotics is on and @CovariantAI leads the way: techcrunch.com/2023/04/04/covariants-robotic-picking-at-n…
-
Pieter Abbeel reposted
Covariant
@CovariantAI
We are excited to announce our funding of $75M to meet growing demand for increased warehouse efficiency and resilience through AI-powered robotic automation.
techcrunch.com/2023/04/04/covariants-robotic-picking-at-n… -
Pieter Abbeel reposted
Covariant
@CovariantAI
@pabbeel on why a #foundationmodel trained on millions of objects from grocery to apparel to pharma – performs best compared to #ai systems specialized for a single industry.
Listen to the full chat with @alexandr_wang of @scale_AI on @therobotbrains: youtu.be/BNqb3Xuv-0AThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Krishna
@jkrshn
Very interesting discussion around hard attention, innate priors etc.
There are atleast a handful of PhD topics in here
Pieter Abbeel
@pabbeel
Super-excited to kick off S3 of @therobotbrains podcast with Yoshua Bengio. We discuss LLMs, Higher-Level Cognition, Causality, GFlow Nets, Responsible AI, Human Creativity.
youtu.be/wdExmzSfw4g -
Pieter Abbeel reposted
Kevin Zakka
@kevin_zakka
Introducing 𝗥𝗼𝗯𝗼𝗣𝗶𝗮𝗻𝗶𝘀𝘁 🎹🤖, a new benchmark for high-dimensional robot control! Solving it requires mastering the piano with two anthropomorphic hands.
This has been one year in the making, and I couldn’t be happier to release it today! Some highlights below:This media couldn't be displayed. See it on the original post. -
Pieter Abbeel
@pabbeel
You had me at "an earnest Orange County bro" :)
(what an excellent cold email rcvd earlier today) -
Pieter Abbeel
@pabbeel
New episode, with @alexandr_wang from @scale_AI!
We discuss Data, Labeling, Foundation Models, LLMs, truthfulness, RLHF, AI for Defense. @therobotbrains
youtu.be/BNqb3Xuv-0APost not available here, view on X
Pieter Abbeel
@pabbeel
Links on other platforms:
Apple: podcasts.apple.com/us/podcast/alexandr-wang-of-scale-ai/i…
Spotify: open.spotify.com/episode/5sOdUdIwM9dsSs2Mvs3i1Y
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5hY2FzdC5jb20v…
Acast: shows.acast.com/the-robot-brains/episodes/alexandr-wang
Pieter Abbeel
@pabbeel
Thanks so much @alexandr_wang for making the time, and a special thank you also to @IndexVentures
and @wandb for supporting the production of the podcast! -
Pieter Abbeel reposted
Aravind Srinivas
@AravSrinivas
We're also excited to launch the Perplexity iOS App today on the App Store:
www.perplexity.ai/iphone
We hope this accelerates the transition from search engines to trustworthy answer engines powered by GPT-x. -
Pieter Abbeel reposted
Aravind Srinivas
@AravSrinivas
Excited to announce that we’ve raised 25.6M$ at a 150M$ valuation led by New Enterprise Associates, along with our seed round lead investors Elad Gil and Nat Friedman. Also honored to have @paultoo and @soleio partner in this round to help us build a well-designed and useful consumer product.
www.perplexity.ai/blog?s=t -
Pieter Abbeel reposted
CoreWeave Developers
@coreweavedevs
Great interview with @chelseabfinn by @pabbeel about Meta-Learning, LLMs and some of the current limitations of AI.
We're delighted to support @therobotbrains podcast!
www.youtube.com/watch?v=ZD15OtMbaNw
Pieter Abbeel
@pabbeel
New @therobotbrains episode is out, with guest @chelseabfinn! We discuss Distribution Shift, Meta-Learning, Editing LLMs, Single-Life RL, and What can AI not (yet) do today?
youtu.be/ZD15OtMbaNw -
Pieter Abbeel reposted
Chelsea Finn
@chelseabfinn
I had fun chatting with Pieter on @therobotbrains podcast! 🤖
Check out the episode for my perspectives on big challenges in AI, research I'm excited about, and other misc topics.
Pieter Abbeel
@pabbeel
New @therobotbrains episode is out, with guest @chelseabfinn! We discuss Distribution Shift, Meta-Learning, Editing LLMs, Single-Life RL, and What can AI not (yet) do today?
youtu.be/ZD15OtMbaNw -
Pieter Abbeel
@pabbeel
New @therobotbrains episode is out, with guest @chelseabfinn! We discuss Distribution Shift, Meta-Learning, Editing LLMs, Single-Life RL, and What can AI not (yet) do today?
youtu.be/ZD15OtMbaNwPost not available here, view on X
Pieter Abbeel
@pabbeel
Links on other platforms:
Apple: podcasts.apple.com/us/podcast/chelsea-finn-how-to-build-a…
Spotify: open.spotify.com/episode/2I4XdB1WUrgvt1a2OERNhC
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/chelsea-finn
Pieter Abbeel
@pabbeel
Thanks so much @chelseabfinn for making the time, and a special thank you also to @IndexVentures and @wandb for supporting the production of the podcast!
-
Pieter Abbeel reposted
Yuqing Du
@d_yuqing
How can we develop more generalisable reward models for agent behaviours?
Excited to share my @deepmind internship project, where we investigate finetuning Flamingo🦩w/ human reward annotations to train success detectors in 3 different domains!
📜arxiv.org/abs/2303.07280
🧵1/ -
Pieter Abbeel reposted
Ken Goldberg
@Ken_Goldberg
First lecture today at 4pm PT: “The Sensorimotor Road to Artificial Intelligence” @UCBerkeley Professor Jitendra Malik, International House and livestream:
www.youtube.com/watch?v=f6fDpKDxpL0&ab_channel=UCBerkeley…
Ken Goldberg
@Ken_Goldberg
Will #chatgpt replace workers? Will it change how we think about thinking? @UCBerkeley scholars who have been studying AI for many years will share their insights in upcoming lectures: www.berkeley.edu/ai/
-
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Last week, we asked ChatGPT who should join @pabbeel for Season 3. So far it's one for one.
If you haven't yet, check out our latest episode with Yoshua Bengio ➡️ youtu.be/wdExmzSfw4g -
Pieter Abbeel
@pabbeel
Super-excited to kick off S3 of @therobotbrains podcast with Yoshua Bengio. We discuss LLMs, Higher-Level Cognition, Causality, GFlow Nets, Responsible AI, Human Creativity.
youtu.be/wdExmzSfw4gPost not available here, view on X
Pieter Abbeel
@pabbeel
Links on other platforms:
Apple: podcasts.apple.com/us/podcast/yoshua-bengio-equipping-ai-…
Spotify: open.spotify.com/episode/2pJuhntZ2k6ZD7jKme4t1c
Amazon: music.amazon.com/podcasts/944100c5-b2f7-4124-84e0-5f74ebe…
Google: music.youtube.com/googlepodcasts
Acast: shows.acast.com/the-robot-brains/episodes/yoshua-bengio
Pieter Abbeel
@pabbeel
A special thank you to @IndexVentures and @wandb for supporting the production of the podcast 🙏🙏🙏
-
Pieter Abbeel reposted
Radial
@RadialCorp
Shipping up to 20,000 parcels per day—and 50,000 during peak—@Radial’s Trade Port 2 #ecommerce #fulfillmentcenter leverages @Covariant’s AI robotics to achieve greater productivity through automation. Watch the video to see innovation in action! www.radial.com/insights/robotic-automation-in-action
-
Pieter Abbeel reposted
UC Berkeley EECS
@Berkeley_EECS
-
Pieter Abbeel reposted
Sherry Yang
@sherryyangML
Review paper on Foundation Models for Decision Making: arxiv.org/abs/2303.04129
Foundation models can characterize various components of decision making, such as states (S), behaviors (A), dynamics (T), task specifiers (R), through generative modeling or representation learning. -
Pieter Abbeel reposted
Aravind Srinivas
@AravSrinivas
Many chrome extension users of @perplexity_ai were interested in just typing "summarize this page for me". So, we decided to just make it even easier by shipping a one-click summary button. Works on public webpages and PDFs, no private information or user data collected. Start using the extension which we envision to be your "search copilot" for web browsing and save you a lot of time in getting up to speed on things! chrome.google.com/webstore/detail/perplexity-ask-ai/hlgbc…
Perplexity
@perplexity_ai
Get to the heart of any article with Perplexity's new one-click summaries! Whether you’re browsing news, articles, or papers, instant answers are just one click away. 📚
Available on our Chrome extension now:
chrome.google.com/webstore/detail/perplexity/hlgbcneanomp… -
Pieter Abbeel reposted
Covariant
@CovariantAI
Discover @RadialCorp's two key requirements for robotic picking automation:
✔️ Ability to retrofit existing infrastructure and footprint
✔️ Integrate seamlessly with their operator workflows and processes
Watch the full video: covariant.ai/resources/radial-leverages-covariant-ai-robo… -
Pieter Abbeel reposted
Hao Liu
@haoliuhl
Humans learn from rich feedback in the form of language. Why not turning all feedback into a sentence to train models?
We propose CoH: Just tell models which ones are not good and which ones are better.
Better than SFT and RLHF on summary and dialogue tasks. -
Pieter Abbeel reposted
Kimin
@kimin_le2
📄Can "learning from human feedback" improve text-to-image models?
I'm excited to share "Aligning Text-to-Image Models using Human Feedback"
📝 arxiv.org/abs/2302.12192
1/N -
Pieter Abbeel
@pabbeel
Absolutely incredible @Astro_Woody (@berkeley_ai PhD, so proud!) that you'll be piloting SpaceX Crew-6 / ISS Expedition 69. Much looking forward to watching your launch this weekend. And what an amazing video to allow people to get to know you better!
youtu.be/QzpQJnoG0cI -
Pieter Abbeel reposted
Zhongyu Li
@ZhongyuLi4
Our goal is to push the limits of the agility of legged robots. Using a single RL-based multi-task policy, a biped Cassie learns to perform various challenging jumps, including standing long jump, high jump, and/or multi-axes jumps, with significant robustness, in the real world.
-
Pieter Abbeel reposted
Tianjun Zhang
@tianjun_zhang
Can we use purely supervised learning for RLHF using large language models? We introduce HIR (Hindsight Instruction Relabeling), which achieves impressive results using FLAN-T5 on hard BigBench tasks!
-
Pieter Abbeel reposted
Perplexity
@perplexity_ai
👋 Say hello to a more personalized browsing experience with our updated Chrome extension! perplexity.ai can now provide answers focused on the page or website you're currently looking at. No more sifting through irrelevant search results:
chrome.google.com/webstore/detail/perplexity/hlgbcneanomp… -
Pieter Abbeel reposted
Yuqing Du
@d_yuqing
How can we encourage RL agents to explore human-meaningful behaviors *without* a human in the loop?
@OliviaGWatkins2 and I are excited to share “Guiding Pretraining in Reinforcement Learning with LLMs”!
📜arxiv.org/abs/2302.06692
🧵1/ -
Pieter Abbeel reposted
Seohong Park
@seohong_park
Can an RL agent autonomously discover complex yet useful behaviors without supervision?
We introduce a novel unsupervised skill discovery method, Controllability-aware Skill Discovery (CSD), that actively seeks *hard-to-achieve* and thus potentially more useful behaviors. -
Pieter Abbeel reposted
Hao Liu
@haoliuhl
We introduce an unsupervised method to align text and image.
Language Quantized AutoEncoders (LQAE) enables few-shot image classification with GPT3 and linear classification of images based on RoBERTa text features.
paper: arxiv.org/abs/2302.00902
code: github.com/lhao499/lqae -
Pieter Abbeel reposted
AIX Ventures
@aixventureshq
We’re hosting an AI event for founders in the Bay Area on 3/11
Details
1️⃣ AIX inc @antgoldbloom, @chrmanning, @pabbeel, @RichardSocher, @shaunbjohnson, & @bianca_caban
2️⃣ Talks
3️⃣ AI++ community
App link below — come join us!
#ai #founders #community #partners #sf #irl -
Pieter Abbeel reposted
The Full Stack
@full_stack_dl
🚀 Come one, come all, to LLM BOOTCAMP 🚀
• Learn best practices and tools for building LLM-powered apps
• Network with other builders and present your projects
• Get cloud credits from OpenAI and other sponsors
• In person, in SF, April 21-22
fullstackdeeplearning.com/llm-bootcamp/
⬇️ -
Pieter Abbeel reposted
Perplexity
@perplexity_ai
Discover and learn faster with Perplexity AI. For any question you have, perplexity.ai will now link to new topics you might be curious about next. Whether you’re reading about people, places, or products, you can now explore and read more simply by clicking:
-
Pieter Abbeel
@pabbeel
Multi-View Masked World Models allow for reliable vision-based robot control even if the camera is being moved. Scientifically this is some really exciting work led by @younggyoseo, also practically really exciting, because hand-eye calibration is such a pain point.
Younggyo Seo
@younggyoseo
Excited to share Multi-View Masked World Models (MV-MWM) that learns multi-view representations and a world model for viewpoint-robust control!
MV-MWM enables zero-shot sim2real transfer with hand-held cameras, only using pixel observations 🚀
sites.google.com/view/mv-mwm -
Pieter Abbeel reposted
AIX Ventures
@aixventureshq
The AIX Ventures team is thrilled to back @itsnancyxu & the @MoonhubAI team on their journey to disrupt the recruiting space with AI.
-
Pieter Abbeel reposted
Perplexity
@perplexity_ai
Perplexity AI is hiring! If you want to find out more about @perplexity_ai and explore the roles we are looking to fill, check out our career page!
Learn more and apply here: www.perplexity.ai/careers -
Pieter Abbeel reposted
Sherry Yang
@sherryyangML
Text-conditioned video generation can serve as universal policies (UniPi) and learn from sim, real, and web-scale videos.
w/ @du_yilun, @hanjundai, @daibond_alpha, @ofirnachum, Josh, Dale, @pabbeel
Paper: arxiv.org/abs/2302.00111
Web: universal-policy.github.io -
Pieter Abbeel reposted
AK
@_akhaliq
ChatGPT rewriting Still D.R.E about reading papers on arxiv
-
Pieter Abbeel reposted
Nat Friedman
@natfriedman
Perplexity is not even six months old and has shipped more product than most companies do in their entire lives. Their latest is a GPT-based search engine that uses webgpt-like techniques to reduce hallucination and to cite its sources.
Perplexity
@perplexity_ai
Announcing a major update to Perplexity Ask: the world’s first conversational search engine! Now, you can read answers with up-to-date sources and ask follow-up questions to dig deeper. In other words, you can chat with your search engine!
Try it out at perplexity.ai -
Pieter Abbeel reposted
Genmo
@genmoai
Announcing Genmo Video, a generative media platform with a new text-to-video model that can generate immersive live artwork from any prompt or any image.
What will you create? 🎨▶️
Free public access: genmo.ai
Discord: discord.com/invite/u7SRpXHhpr
👇1/nThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Perplexity
@perplexity_ai
Announcing a major update to Perplexity Ask: the world’s first conversational search engine! Now, you can read answers with up-to-date sources and ask follow-up questions to dig deeper. In other words, you can chat with your search engine!
Try it out at perplexity.ai -
Pieter Abbeel reposted
jack
@jack
This is great
Perplexity
@perplexity_ai
Introducing Bird SQL, a Twitter search interface that is powered by Perplexity’s structured search engine. It uses OpenAI Codex to translate natural language into SQL, giving everyone the ability to navigate large datasets like Twitter.
www.perplexity.ai/sql -
Pieter Abbeel reposted
Perplexity
@perplexity_ai
Introducing Bird SQL, a Twitter search interface that is powered by Perplexity’s structured search engine. It uses OpenAI Codex to translate natural language into SQL, giving everyone the ability to navigate large datasets like Twitter.
www.perplexity.ai/sql -
Pieter Abbeel reposted
Mezli
@mezli_med
What's Mezli like behind the curtain? We've mentioned before that we've got some sensitive IP we're trying to keep under wraps, but here's a glimpse of what's going on inside: a little food factory, an autonomous assembly line, with bowls all made to order.
-
Pieter Abbeel reposted
Covariant
@CovariantAI
This week at @FortuneMagazine's #brainstormai, Covariant’s own, @pabbeel explains our approach to building our generalized #ai platform, the Covariant Brain.
And how this #foundationmodel provides more robust pick and place performance across all industries. -
Pieter Abbeel reposted
Perplexity
@perplexity_ai
Announcing Perplexity Ask, a new search interface that uses OpenAI GPT 3.5 and Microsoft Bing to directly answer any question you ask.
www.perplexity.ai
discord.com/invite/kWJZsxPDuX -
Pieter Abbeel reposted
Karol Hausman
@hausman_k
We'll have a set of practical implementation talks (including tricks in common RL algorithms):
- ChatGPT author @johnschulman2 on PPO
- @danijarh on Dreamer
- @kristianhartika on SAC
- @aviral_kumar2 and @ikostrikov on CQL and IQL -
Pieter Abbeel reposted
Karol Hausman
@hausman_k
There will be also🔥talks from our opinion speakers:
@UMassScott on how RL should conduct more scientific experiments, @scychan_brains on how RL algos should reflect our highly non-uniform world, and @natashajaques on how RL should be more social. -
Pieter Abbeel reposted
Karol Hausman
@hausman_k
Deep RL workshop @NeurIPSConf starts on Friday!
Hear invited talks from @tobigerstenberg on counterfactual simulation of causal judgments, @j_foerst on opponent-shaping in games, @IMordatch on sequence modeling, @yayitsamyzhang on learning generalist agents. Don't miss out! 🧵 -
Pieter Abbeel reposted
Wojciech Zaremba
@woj_zaremba
Many might not know.
@johnschulman2 is the main person driving ChatGPT, and I would like to congratulate him and the team!
Worth to follow. -
Pieter Abbeel reposted
Hao Liu
@haoliuhl
In our #neurips2022 work, we explore the generality of masked token prediction for generalizable and flexible reinforcement learning.
A 🧵 on the paper arxiv.org/abs/2211.12740 -
Pieter Abbeel reposted
Hao Liu
@haoliuhl
How to pretrain large language-vision models to help seeing, acting, and following instructions?
We found that using models jointly pretrained on image-text pairs and text-only corpus significantly outperforms baselines.
A 🧵 on the paper InstructRL arxiv.org/abs/2210.13431 -
Pieter Abbeel
@pabbeel
very exciting, congratulations @RaquelUrtasun!
TechCrunch
@TechCrunch
Waabi unveils its first generation self-driving truck tcrn.ch/3hAP6Cf by @rebeccabellan
-
Pieter Abbeel reposted
Jack Clark
@jackclarkSF
I just bought myself a load of @arxiv merch for Christmas - and you can too! www.bonfire.com/store/arxiv-official/
This is a great way to support one of the most important open infrastructures of the AI ecosystem. Plus, you'll be swagged out for the festive season. -
Pieter Abbeel reposted
Zongheng Yang
@zongheng_yang
Introducing SkyPilot:
Run ML and Data Science jobs on any cloud, with massive cost savings.
🚀 Run jobs on any cloud
⏰ Get GPU/TPU/CPU in 1 click
💵 Reduce > 3x cost
skypilot.co
Read blog: medium.com/@zongheng_yang/skypilot-ml-and-data-science-on…
🧵1/ -
Pieter Abbeel reposted
Stephen James
@stepjamUK
StereoPose! Our new stereo RGB framework for category-level object pose estimation that works incredibly well for transparent objects!
The magic sauce? Back-view normalized object coordinate space (NOCS)! More below ⬇️.
📜 arxiv.org/abs/2211.01644
Lead: Kai Chen, @CUHKofficial -
Pieter Abbeel reposted
Alex Kendall
@alexgkendall
This year we’ve seen many incredible AI breakthroughs from very large foundation models:
CLIP = image/language
Stable Diffusion = caption2image
GPT-3 = language
Whisper = speech2text
etc
But what does this mean for robotics and autonomous driving? -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
It’s not everyday that @BillGates shares his love for our podcast! 🤩
-
Pieter Abbeel reposted
Hao Liu
@haoliuhl
Can language model pretraining be even better?
Our paper shows that by randomly masking input tokens during pretraining, the zero-shot, few-shot, and fine-tuning performance can be significantly improved.
arxiv.org/abs/2210.13432
🧵 -
Pieter Abbeel reposted
Mike Dauber
@dauber
Application of LLMs are truly remarkable, but AI is going to be everywhere. Incredible that this capability is now <$10 for sample volume. In volume I’m sure it’ll soon be <$1.
Pete Warden
@petewarden
I built a face-following pan/tilt stand using a person sensor, here's a @Hacksterio project guide:
www.hackster.io/petewarden/face-following-pan-tilt-stand-…
This little fella has a lot of personality! -
Pieter Abbeel reposted
Amber Xie
@amberxie_
Excited to release our CoRL 2022 work, Sim-to-Seg: end-to-end RL w/ sim-to-real transfer through learned image segmentations! 🚗⛰
Work done with @johnrso_, @NASAJPL, @pabbeel @stepjamUK
📜: arxiv.org/abs/2210.14721
🌐: sites.google.com/view/sim2segcorl2022/homeThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Bill Gates
@BillGates
Thanks for having me! Seeing the universal Al Robotics platform in action was incredible.
-
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Hi friends! It's that time of year again...reply with your Season 3 guest suggestions. Who's doing the most interesting work in #ai and #robotics that we need to know?
If you've sent a suggestion via email, please know that those folks are being considered as well! 🤓 -
Pieter Abbeel reposted
Covariant
@CovariantAI
Thanks to @BillGates for stopping by to experience the future of AI robotic automation!
Bill challenged the Covariant Brain, our AI Robotics platform, to pick chaotically arranged items. The Covariant Brain did not disappoint, directing the robots to pick with accuracy and ease! -
Pieter Abbeel
@pabbeel
This was a fun time, honored @BillGates stopped by to experience the future of AI robotic automation!
Covariant
@CovariantAI
Thanks to @BillGates for stopping by to experience the future of AI robotic automation!
Bill challenged the Covariant Brain, our AI Robotics platform, to pick chaotically arranged items. The Covariant Brain did not disappoint, directing the robots to pick with accuracy and ease! -
Pieter Abbeel reposted
Ofir Nachum
@ofirnachum
Despite all the buzz about decision transformers, it's well-known they can be unboundedly suboptimal in stochastic envs. The issue is due to conditioning the policy on return, a highly stochastic quantity. We introduce "dichotomy of control" to solve this: arxiv.org/abs/2210.13435
-
Pieter Abbeel reposted
Scale
@scale_AI
How fast can robots learn? 🤖
💡Find out in our next #transformx session with @pabbeel, Co-Founder of @CovariantAI, where he will discuss paving the path to generalizable robotics.
🛑LIVE: scl.ai/3F0gK5y -
Pieter Abbeel reposted
Covariant
@CovariantAI
Revisit @therobotbrains conversation with @pabeel and @KNAPP’s VP of Innovation, Peter Puchwein for an explanation of why #ai is necessary to deploy #industrialrobotics in the real world.
Check out the highlight: youtu.be/5wyP0liYwno -
Pieter Abbeel reposted
Radical Ventures
@radicalvcfund
What does it take to found a successful AI company? Radical's Co-Founder @JordanJacobs10 asked AI roboticist and multi-time founder @pabbeel at the #aifounders Master Class kick-off last week. Read his response in #radicalreads: radical.vc/radical-reads-aifmc-pieter-abbeel/
on.soundcloud.com/zRmjf -
Pieter Abbeel reposted
Ademi Adeniji
@AdemiAdeniji
Excited to announce Intrinsic Reward Matching (IRM) - a novel and efficient algorithmic framework unifying unsupervised RL skill pretraining and downstream learning! w/ @amberxie_ and @pabbeel
📄arxiv.org/pdf/2210.07426.pdf
👩💻github.com/ademiadeniji/irm
(🧵) -
Pieter Abbeel
@pabbeel
Thanks so much to @radicalvcfund for hosting me tomorrow/Wednesday in the AI Founders Master Class. Much looking forward to it, as well as learning from @drfeifei, @aidangomezzz, @RaquelUrtasun, @RichardSocher in the next sessions!
Radical Ventures
@radicalvcfund
If you are an AI researcher interested in entrepreneurship, you can learn more about the AI Founders Master Class and registration in #radicalreads:
radical.vc/radical-reads-ai-founders-master-class/ (4/4) -
Pieter Abbeel
@pabbeel
Congratulations @foxglovedev, much looking forward to what's ahead!
And of course thankful to be part of this journey.
Foxglove
@foxglove
Today, Foxglove is thrilled to announce our $15M Series A funding, led by @Sethwinterroth of @EclipseVentures 🎉
Thank you for following our journey – it is just beginning 🍾
foxglove.dev/blog/foxgloves-15m-series-a-and-the-missing-…
@OpenRoboticsOrg @svrobo @AmplifyPartners #robotics #robots #datavisualization -
Pieter Abbeel reposted
Wilson Yan
@wilson1yan
Excited to announce TECO, an efficient video prediction model that can generate long, temporally consistent video for complex datasets in 3D scenes such as DMLab, Minecraft, Habitat, and real-world video from Kinetics!
📜arxiv.org/abs/2210.02396
🌐wilson1yan.github.io/teco
(🧵) -
Pieter Abbeel reposted
Danijar Hafner
@danijarh
Video prediction has seen great progress recently but long videos are still inconsistent, e.g. when moving around 3D scenes. I'm excited to share Temporally Consistent Video Transformer (TECO), a scalable transformer that substantially improves learning of long dependencies! 🚀
Wilson Yan
@wilson1yan
Excited to announce TECO, an efficient video prediction model that can generate long, temporally consistent video for complex datasets in 3D scenes such as DMLab, Minecraft, Habitat, and real-world video from Kinetics!
📜arxiv.org/abs/2210.02396
🌐wilson1yan.github.io/teco
(🧵) -
Pieter Abbeel reposted
Stephen James
@stepjamUK
@corl_conf Oral!
Self-supervised in-the-wild video pre-training for real-world robotic tasks!🤖
Our 307M parameter vision transformer outperforms CLIP, ImageNet pre-training, and training from scratch!
📜arxiv.org/abs/2210.03109
🌐tetexiao.com/projects/real-mvp
🧠github.com/ir413/mvp -
Pieter Abbeel reposted
Covariant
@CovariantAI
Concerned about pursuing capital-heavy strategies?
In this @BiznessReporter video, Covariant co-founders @pabbeel and @peterxichen share how advancements in #airobotics can help you optimize your warehouse — increasing throughput without adding cost.
business-reporter.co.uk/technology/better-not-bigger-addr… -
Pieter Abbeel reposted
Angjoo Kanazawa
@akanazawa
Releasing 🌟nerfstudio🌟, a plug-and-play python library to easily create your own NeRFs!
@nerfstudioteam is a contributor friendly open-source repo with a realtime web viewer that makes it easy to make cool videos 📽️
github.com/nerfstudio-project/nerfstudio
docs.nerf.studio #nerfstudio 1/ -
Pieter Abbeel reposted
Jonathan Ho
@hojonathanho
Excited to announce Imagen Video, our new text-conditioned video diffusion model that generates 1280x768 24fps HD videos! #imagenvideo
imagen.research.google/video/
Work w/ @wchan212 @Chitwan_Saharia @jaywhang_ @RuiqiGao @agritsenko @dpkingma @poolio @mo_norouzi @fleet_dj @TimSalimans -
Pieter Abbeel
@pabbeel
We were so fortunate to have @ylecun visit @UCBerkeley yesterday. Here is the talk recording:
www.youtube.com/watch?v=VRzvpV9DZ8Y -
Pieter Abbeel reposted
Covariant
@CovariantAI
In this fascinating interview with @ODSC, Covariant co-founder and Chief Scientist, @pabbeel, explains how and why Deep Reinforcement Learning is such a central aspect to developing modern AI.
www.youtube.com/watch?v=2s3uyKvfGF4&hss_channel=tw-106095… -
Pieter Abbeel reposted
Karol Hausman
@hausman_k
Are you working on @iclr_conf deadline and you need a few more days to finish up the experiments for Deep RL workshop @NeurIPSConf? We got you!
We're extending the deadline to October 3rd! Good luck with your submissions!
Karol Hausman
@hausman_k
Super excited to announce this year’s Deep RL workshop @NeurIPSConf 🎉🎉🎓🎓
website: sites.google.com/corp/view/deep-rl-workshop-neurips-2022
submission deadline: September 22nd, 11:59 PM PST
Some exciting changes that we made this year in 🧵👇 -
Pieter Abbeel reposted
Lukas Biewald
@l2k
“Play is the equivalent of pre-training for RL agents.” – @pabbeel on GD.
▶️Watch the full episode here: wandb.ai/wandb_fc/gradient-dissent/reports/Pieter-Abbeel-… -
Pieter Abbeel reposted
Covariant
@CovariantAI
Off to the races. We're lining up the final robotic deployments before peak season kicks off.
Are you ready?
#warehousing #airobotics -
Pieter Abbeel reposted
Covariant
@CovariantAI
In this @FreightWaves article, Covariant CEO @peterxichen offers insight into optimizing warehouse throughput to scale up or down to meet fluctuating demand without risky capital outlays.
www.freightwaves.com/news/viewpoint-unpredictable-markets… -
Pieter Abbeel
@pabbeel
It's a great time to build better, not bigger, warehouses
www.freightwaves.com/news/viewpoint-unpredictable-markets… -
Pieter Abbeel reposted
ODSC (Open Data Science Conference) AI
@_odsc
On September 16th, we will interview ODSC West speaker @pabbeel – PhD, Director | Co-Director at Berkeley Robot Learning Lab | Berkeley Artificial Intelligence (BAIR) Lab. He will lead a conversation on "Foundations of Deep Reinforcement Learning". share.hsforms.com/1kM5rwby0TLiIANK294eGxA13zdw?hss_channe…
-
Pieter Abbeel
@pabbeel
with real-world robotic experiments still more costly and time-consuming than simulated experiments, really exciting to see these major mujoco updates!
Google DeepMind
@GoogleDeepMind
Introducing the #mujoco Menagerie, a collection of robot models curated by DeepMind: dpmd.ai/menagerie 1/
-
Pieter Abbeel reposted
Raluca Ada Popa
@ralucaadapopa
UC Berkeley researchers have an impressive record of contributing open source, and not just that — but also sweating the extra mile to make it into an easy to use product that can address real problems users face. I am a strong believer in this. @opaquesys www.forbes.com/sites/brucerogers/2022/09/01/berkley-acade…
-
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Playing around with @StableDiffusion today got us like 🤯 huggingface.co/spaces/stabilityai/stable-diffusion
-
Pieter Abbeel reposted
AIX Ventures
@aixventureshq
A thought provoking keynote by @erikbryn, with intros from co-founders @RichardSocher & Shaun Johnson, about the economic implications of AI automation at our AIX Ventures Launch Event earlier this month.
We hope you enjoy it and leave comments below!
www.youtube.com/watch?v=w4zIZL5QiZk -
Pieter Abbeel reposted
Association for Computing Machinery
@TheOfficialACM
In this week's People of ACM profile, we feature @pabbeel, a Professor at @Berkeley_EECS, Co-Founder of the AI robotics company @CovariantAI, and recipient of 2021 ACM Prize in Computing. Read the full interview here: www.acm.org/articles/people-of-acm/2022/pieter-abbeel
-
Pieter Abbeel reposted
Zico Kolter
@zicokolter
Announcement: This Fall @tqchenml and I are releasing a free online version of our Deep Learning Systems course! Short description is you build a deep learning framework from the ground up.
Sign up at course website: dlsyscourse.org
Video teaser: www.youtube.com/watch?v=qbJqOFMyIwgThis media couldn't be displayed. See it on the original post. -
Pieter Abbeel
@pabbeel
This should be really fun and delicious, and robots!
Mezli
@mezli_med
At long last, we're launching! Get stoked for our Grand Opening at #sparksocial in #sanfrancisco, Sunday August 28. Come by and celebrate with us!
Huge thanks to @eatersf, especially Lauren & Albert, for this awesome piece.
sf.eater.com/2022/8/17/23308389/mezli-robot-restaurant-op… -
Pieter Abbeel reposted
Berkeley AI Research
@berkeley_ai
Check out the upcoming Ray Summit, organized by BAIR alumni at Anyscale!
anyscale.com/ray-summit-2022
Robert Nishihara
@robertnishihara
Details here for
🌁 In-person networking in SF
🎒 3 in-depth Ray training sessions
⚙️ 40+ technical sessions and lightning talks
🥳 Welcome party at the Exploratorium
www.anyscale.com/ray-summit-2022 -
Pieter Abbeel reposted
Andrej Karpathy
@karpathy
!!!! Ok I recorded a (new!) 2h25m lecture on "The spelled-out intro to neural networks and backpropagation: building micrograd" www.youtube.com/watch?v=VMj-3S1tku0 .
This is the culmination of about 8 years of obsessing about the best way to explain neural nets and backprop. -
Pieter Abbeel reposted
Gantry
@gantry_ml
📜 Production ML papers to know 📜
In this series, we'll cover some of the most important papers to know as a builder of ML-powered products.
Up first, we have one of the all time classics: the paper that predicted MLOps back in 2014.
youtu.be/tjgvYpnuTK4 -
Pieter Abbeel reposted
Josh Tobin
@josh_tobin_
I thought it could be fun to share some of my favorite papers on production ML in a more accessible format.
Check out the first version, where we cover one of the all-time classics: the high-interest credit card paper:
Gantry
@gantry_ml
📜 Production ML papers to know 📜
In this series, we'll cover some of the most important papers to know as a builder of ML-powered products.
Up first, we have one of the all time classics: the paper that predicted MLOps back in 2014.
youtu.be/tjgvYpnuTK4 -
Pieter Abbeel reposted
Covariant
@CovariantAI
In the first installment of our new series, Automation, Upgraded, we take a look at a way to take that efficiency a step further: by turning that goods-to-person system into a goods-to-robot system. #airobotics #artificialintelligence #warehousing
covariant.ai/news/automation-upgraded-robotic-goods-to-pe… -
Pieter Abbeel reposted
Covariant
@CovariantAI
Covariant is honored to be named the 2022 @AI_Breakthrough Awards Best Overall Robotics Company! The judges evaluated companies based on innovation, performance, ease of use, functionality, value, and impact.
aibreakthroughawards.com/2022-winners/?hss_channel=tw-106… -
Pieter Abbeel
@pabbeel
Very excited for the 2022 edition of the #neurips Deep RL workshop! A few fun changes, see below. Also, submission deadline: Sep 22
Karol Hausman
@hausman_k
Super excited to announce this year’s Deep RL workshop @NeurIPSConf 🎉🎉🎓🎓
website: sites.google.com/corp/view/deep-rl-workshop-neurips-2022
submission deadline: September 22nd, 11:59 PM PST
Some exciting changes that we made this year in 🧵👇 -
Pieter Abbeel reposted
Qiyang (Colin) Li
@qiyang_li
Check out our latest work on generative modeling with Adaptive Categorical Discretization (AdaCat)!
AdaCat generalizes uniform discretization and can improve existing autoregressive models on density estimation for tabular data, images, audio, and trajectories in RL. Thread: 1/N -
Pieter Abbeel reposted
Ajay Jain
@ajayj_
New work on autoregressive generative models! We improve the expressiveness of your favorite continuous autoreg models like Trajectory Transformer, WaveNet and Image GPT with Adaptive Categorical Discretization. See below for code. Paper at #uai2022 colinqiyangli.github.io/adacat/
Qiyang (Colin) Li
@qiyang_li
Check out our latest work on generative modeling with Adaptive Categorical Discretization (AdaCat)!
AdaCat generalizes uniform discretization and can improve existing autoregressive models on density estimation for tabular data, images, audio, and trajectories in RL. Thread: 1/N -
Pieter Abbeel reposted
TechXplore
@TechXplore_com
DayDreamer: An algorithm to quickly teach robots new behaviors in the #realworld techxplore.com/news/2022-07-daydreamer-algorithm-quickly-…
-
Pieter Abbeel reposted
Charles C Onu
@onucharlesc
At the heart of the Ubenwa team is a deep desire to save newborn lives and reduce long-term disability using the power of AI.
So I'm very excited to share that we have raised US$2.5M (CA$3.2M) to accelerate this mission!
techcrunch.com/2022/07/25/can-a-babys-cry-identify-neurol… -
Pieter Abbeel reposted
Supply Chain Xchange
@theSCXchange
Catch the latest episode of Supply Chain in the Fast Lane. We speak to Anu Saha of @CovariantAI about #artificialintelligence, #robotics, and applications that are driving efficiencies in the #supplychain. Listen at www.supplychainquarterly.com/podcasts #ai #logistics
-
Pieter Abbeel reposted
Berkeley AI Research
@berkeley_ai
Check out this @TechCrunch video highlighting cool robot learning research led by @stepjamUK @AleEscontrela @danijarh and @philippswu.
@pabbeel @Ken_Goldberg @Berkeley_EECS
youtu.be/h8AUJwPdTIE -
Pieter Abbeel reposted
Alejandro Escontrela
@alescontrela
The MIT @techreview published an excellent summary of our work today!
We leverage learned world models to teach a robot dog to walk in under one hour without simulators or priors
S/o to my amazing team: @philippswu @danijarh @Ken_Goldberg @pabbeel
www.technologyreview.com/2022/07/18/1056059/robot-dog-ai-… -
Pieter Abbeel reposted
Danijar Hafner
@danijarh
Excited to share our article on quickly training robots in the real world through world models in MIT Technology Review! Thanks @philippswu @AleEscontrela @Ken_Goldberg @pabbeel @Melissahei @techreview @LerrelPinto @UoE_Agents
www.technologyreview.com/2022/07/18/1056059/robot-dog-ai-… -
Pieter Abbeel reposted
Berkeley AI Research
@berkeley_ai
Congratulations to @Berkeley_EECS PhD @Astro_Woody on joining the @SpaceX Crew-6 mission as pilot! Woody completed his PhD with @pabbeel in 2013 and has been a @NASA astronaut since 2017.
engineering.berkeley.edu/news/2022/07/to-infinity-and-bey… -
Pieter Abbeel reposted
Stephen McAleer
@McaleerStephen
We introduce a simple ensemble method to reduce variance in value estimation. MeanQ maintains an ensemble of k Q networks and then takes the max of the average as a target value in the standard TD loss. To appear at #icml2022! (link: royf.org/pub/pdf/Liang2022Reducing.pdf)
-
Pieter Abbeel reposted
Danijar Hafner
@danijarh
We just released a Google AI Blog post on Director that explains how the method works and what it can do! 🤖
Google AI
@GoogleAI
Artificial agents often have difficulty managing complex real-world tasks. To address this shortcoming, we introduce a hierarchical #rl approach that trains a manager policy to propose achievable subgoals to be executed by a worker policy. Learn more at goo.gle/3O6LtyM
-
Pieter Abbeel reposted
Danijar Hafner
@danijarh
The full training run of the A1 quadruped robot learning to walk from scratch in the real world in 1 hour! Made possible by training a world model online and planning inside of it. Excited to see what we can do next with this! @philippswu @AleEscontrela @Ken_Goldberg @pabbeel
This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Ryan Hoque
@ryan_hoque
Companies like @Waymo & @Amazon use remote humans to manage fleets of robots for self-driving & logistics. We introduce new formalism, algorithms, and open-source @NVIDIA Isaac Gym benchmarks for “Interactive Fleet Learning” (IFL).
w/ @ken_goldberg, @pabbeel, @berkeley_ai (1/8) -
Pieter Abbeel reposted
Victor Butoi
@ion_barrel
@pabbeel taking that Robot Brains podcast with Hinton to heart with this new work on Learning via Masked Autoencoders arxiv.org/abs/2206.14244
-
Pieter Abbeel reposted
Gradescope
@gradescope
The Gradescope website has a new look!
The new Gradescope branding brings our website close in line with the larger Turnitin branding. Head over to gradescope.com to check it out!
Stay tuned for branding updates to the Gradescope web and mobile apps later this year. -
Pieter Abbeel reposted
Younggyo Seo
@younggyoseo
Excited to share Masked World Models for Visual Control!
Inspired by MAE and World Models, we train an autoencoder with convolutional feature masking and reward prediction, then train a dynamics model in the latent space of the autoencoder.
arxiv.org/abs/2206.14244
Thread 🧵 -
Pieter Abbeel reposted
AIX Ventures
@aixventureshq
Today we're thrilled to announce @suchisaria has joined AIX Ventures as its newest Investing Partner. Check out @RichardSocher's announcement for more info.
Welcome, Suchi!
www.aixventures.com/announcements/welcoming-our-newest-in… -
Pieter Abbeel reposted
Danijar Hafner
@danijarh
A dream come true! We introduce DayDreamer, where we apply world models for fast end-to-end learning on 4 physical robots, without simulators.
We learn quadruped walking from scratch in 1 hour. We also learn to pick & place balls directly from pixels and sparse rewards 🤖🌏👇 -
Pieter Abbeel reposted
Philipp Wu
@philippswu
Excited to share our paper DayDreamer! We test the Dreamer algorithm on 4 robots and learn from scratch with no simulators. Really enjoyed collaborating with @AleEscontrela and @danijarh to make this a reality. Thanks to @Ken_Goldberg and @pabbeel for their guidence and support!
Danijar Hafner
@danijarh
A dream come true! We introduce DayDreamer, where we apply world models for fast end-to-end learning on 4 physical robots, without simulators.
We learn quadruped walking from scratch in 1 hour. We also learn to pick & place balls directly from pixels and sparse rewards 🤖🌏👇 -
Pieter Abbeel reposted
Alejandro Escontrela
@alescontrela
Excited to announce DayDreamer, an application of learned world models to robot learning without simulators!
DayDreamer enables a robot dog to learn to walk in one hour without resets, simulators, or demonstrations, as well as visual pick and place and navigation tasks
thread🧵 -
Pieter Abbeel reposted
Covariant
@CovariantAI
Covariant has been named the 2022, Best Overall Robotics Company by the @AI_Breakthrough awards, which recognize the top companies, technologies and products in AI today. The list is full of good company, and we’re honored to be here!
aibreakthroughawards.com/2022-winners/?hss_channel=tw-106… -
Pieter Abbeel reposted
Russ Salakhutdinov
@rsalakhu
1/4 I was watching @geoffreyhinton interview w/t @pabbeel - it reminded me of a couple fun stories about Geoff and how he got me into deep learning.
Back in 2005, I was not thinking about PhD & had a good career as ML engineer. One morning, on my way to work I bumped into Geoff. -
Pieter Abbeel reposted
The Full Stack
@full_stack_dl
We are excited to announce FSDL 2022!
✨ New lectures on continual learning, working with LLM's, and ML project management
💪 All lectures and labs updated for 2022
🗣 Discord-based cohort experience
🔜 Starting August 8th
Register today at fullstackdeeplearning.com/course/ -
Pieter Abbeel reposted
Nando de Freitas
@NandoDF
@geoffreyhinton on the need for learning algorithms for spiking neural networks, consciousness as a pre-scientific concept, and other interesting thoughts. Thanks @pabbeel for another informative and entertaining interview. This podcast rocks!
The Robot Brains Podcast
@therobotbrains
He has answered your questions... now, tune in to hear @geoffreyhinton answer @pabbeel's 10 signature questions in our final "On The DL" segment of the season! 🤖🧠 youtu.be/iBuAobuBoxQ
-
Pieter Abbeel reposted
Danijar Hafner
@danijarh
The Visual Pin Pad suite allows studying sparse long-horizon tasks decoupled from visual complexity. The agent controls the black square and needs to step on all color pads in the correct order to get the sparse reward. Director also solves all four difficulties of this task
-
Pieter Abbeel reposted
Danijar Hafner
@danijarh
Excited to share Director, a practical, general, and interpretable reinforcement learning algorithm for learning hierarchical behaviors from pixels!
Director explores and solves long-horizon tasks with very sparse rewards by breaking them down into internal subgoals.
Thread 👇 -
Pieter Abbeel reposted
Andrew Davison
@AjdDavison
I really enjoyed this interview; lots of good reminders about what's important and fun about curiosity-driven research.
Pieter Abbeel
@pabbeel
Chatting with @geoffreyhinton was really inspiring, so excited to get to share our conversation:
Apple: podcasts.apple.com/us/podcast/geoff-hinton-on-revolutioni…
Spotify: spoti.fi/3GBDpDM
Google: music.youtube.com/googlepodcasts
Youtube: youtu.be/2EDP4v-9TUA -
Pieter Abbeel reposted
Stephen James
@stepjamUK
New! Patch-based Object-centric Video Transformer.
We use object-centric information (bounding boxes) as a compressed representation for videos, giving improved computational efficiency on long-horizon video prediction.
Lead: Wilson Yan w/ Ryo O, @pabbeel
arxiv.org/abs/2206.04003 -
Pieter Abbeel reposted
ZenML Labs
@zenml_io
Our @wandb 💘 ZenML - webinar will happen today 9am PT / 6pm CET! We will transition a W&B tracking workflow from experimentation to production with ZenML pipelines in a live demo. 🤗 We can't wait!
-> join us here: webinars-zenml.wandb.ai
#opensource #mlops -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
In part two of our season finale, @geoffreyhinton answers your questions on AGI, the human brain, the next milestones in #deeplearning, and much more. Thank you, Geoff, for a very special close to The Robot Brains' Season Two - what a privilege! youtu.be/4Otcau-C_Yc
-
Pieter Abbeel reposted
Covariant
@CovariantAI
Check out the latest article from @bob_tita and @WSJ as they discuss robotic automation, the labor challenge, and the companies improving their supply chain resilience in 2022.
#artificialintelligence #warehouseautomation
www.wsj.com/articles/robots-pick-up-more-work-at-busy-fac… -
Pieter Abbeel reposted
Josh Tobin
@josh_tobin_
As easy as it's become to train ML models, it's still way too hard to get them to work well in real products with real users.
We raised a Series A @gantry_ml to solve this problem.
I wrote about the raise and where we're going with the product: gantry.io/blog/introducing-gantry/ -
Pieter Abbeel reposted
Kyle Wiggers
@Kyle_L_Wiggers
Gantry launches out of stealth to help data scientists keep AI models fresh techcrunch.com/2022/06/07/crunch/
-
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
A little sneak peek into questions answered by @geoffreyhinton:
are you concerned with AI becoming too successful?
what is the connection between mania and genius?
what is next in AI?
what should PhD students focus on?
how conscious do you think today's neural nets are? -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Tune in Wednesday to finally hear @geoffreyhinton answer YOUR questions in part two of our season finale. 🤖🧠
Pieter Abbeel
@pabbeel
Exciting announcement: in my @therobotbrains conversation with @geoffreyhinton (yes!!!!!!!!!), there will be some time for (prepared) audience questions. By Tuesday, post/like questions you have for Geoff in reply to this tweet, and I'll try to incorporate into our conversation.
-
Pieter Abbeel reposted
Ken Goldberg
@Ken_Goldberg
Very interesting to hear @geoffreyhinton questioning backpropagation, advocating for local reward functions, describing adversarial examples as human error, and recounting the history of t-SNE! (BTW: skeptics are quite different than deniers;)
Pieter Abbeel
@pabbeel
Chatting with @geoffreyhinton was really inspiring, so excited to get to share our conversation:
Apple: podcasts.apple.com/us/podcast/geoff-hinton-on-revolutioni…
Spotify: spoti.fi/3GBDpDM
Google: music.youtube.com/googlepodcasts
Youtube: youtu.be/2EDP4v-9TUA -
Pieter Abbeel
@pabbeel
Always an honor to be featured back in Belgium. Thank you @RoelVerrycken for such a great conversation and article in @tijd!
www.tijd.be/dossier/ontbijt/pieter-abbeel-hoe-bouwen-we-i… -
Pieter Abbeel reposted
utku
@utkuevci
Recommended! “I was used to just having everybody else being wrong and obviously wrong... You need the faith… in science to be willing to work on stuff just because its obviously right even though everyone else says its nonsense” podcasts.apple.com/ca/podcast/the-robot-brains-podcast/id… @geoffreyhinton @pabbeel
-
Pieter Abbeel reposted
Ilya Sutskever
@ilyasut
An extremely interesting interview with my PhD advisor @geoffreyhinton
The Robot Brains Podcast
@therobotbrains
Honored to have @geoffreyhinton as the final guest of Season 2, discussing his journey in academia to Google Brain, how existing neural nets and BP operates differently than how the brain works, and why it's better to grow computers than manufacture them. youtu.be/2EDP4v-9TUA
-
Pieter Abbeel
@pabbeel
Chatting with @geoffreyhinton was really inspiring, so excited to get to share our conversation:
Apple: podcasts.apple.com/us/podcast/geoff-hinton-on-revolutioni…
Spotify: spoti.fi/3GBDpDM
Google: music.youtube.com/googlepodcasts
Youtube: youtu.be/2EDP4v-9TUAPost not available here, view on X
Pieter Abbeel
@pabbeel
- Transition from pure academia to Google
- The secret auction of Hinton’s company at NeurIPS
- Hinton’s start in psychology and carpentry
- Why computers should be grown rather than manufactured
- The function of sleep and Boltzmann Machines
- Need for negative data
Pieter Abbeel
@pabbeel
Also, not only is this episode longer than most already, stay tuned for part 2 next week, where we'll cover a bunch of the audience questions shared with us on twitter!
Yes, @geoffreyhinton isn't only genius, but was also incredibly generous with his time! 🙏 -
Pieter Abbeel reposted
Hao Liu
@haoliuhl
Excited to share M3AE, a simple but effective model for multimodal representation learning.
TLDR: M3AE learns a unified encoder for both vision and language from both paired image-text data as well as unpaired data.
arxiv.org/abs/2205.14204 w/ @YoungGeng
Summary thread:
[1/N] -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Our final guest of Season 2 is someone who's been mentioned on the show quite a few times already….and for a very good reason.
-
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Fresh off their $2B valuation, the CEO of @huggingface, @ClementDelangue joins us to discuss the power of open source collaboration and how #nlp has developed from a niche #machinelearning topic to arguably the biggest domain in ML today.🤗youtu.be/db_ltgsCR54
-
Pieter Abbeel
@pabbeel
Very exciting (and impressive!) position as the Founding Director of the Dyson Robot Learning Lab @stepjamUK, congratulations!!! Will miss you here at @UCBerkeley..
Stephen James
@stepjamUK
Following @Dyson's big robotics reveal...
I'm honoured and excited to be appointed Director of Robot Learning and founding a new Dyson Robot Learning Lab in London! We are hiring research scientists, robotics engineers, game devs, interns.
Log interest: forms.gle/5CqzpZ9kJDeSoGgf7This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
Sherry Yang
@sherryyangML
What does "Learn principles, not formulas. Understand, do not memorize” mean for autonomous agents?
Chain of Thought Imitation with Procedure Cloning!
ArXiv arxiv.org/abs/2205.10816
Code github.com/google-research/google-research/tree/master/pr…
Site sites.google.com/view/procedure-cloning
w/ Dale @pabbeel @ofirnachum -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
This week's guest @ClementDelangue shares how @huggingface evolved from an "artificial bestie" chatbot to one of the fastest-growing open source AI platforms, used by over 10,000 companies. Full episode airs Wednesday!🤗
This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
PSA: Looking for a robot that removes garden weeds. 🤖🌱 Watch this week's "On The DL" to learn more about our guest @amitag's green thumb, passion for politics and more! youtu.be/qXanpHVIyLU
-
Pieter Abbeel reposted
THE YES
@theyes
@amitag CTO of @theyes joins @therobotbrains Podcast to share how the company uses ML to create millions of unique stores daily for each one of their users. @pabbeel #makingaifashionable www.therobotbrains.ai/who-is-amit-agarwhal
-
Pieter Abbeel reposted
THE YES
@theyes
@therobotbrains featuring @amitag and @pabbeel exploring how AI is transforming fashion e-commerce going live Wednesday!! Not to be missed!
The Robot Brains Podcast
@therobotbrains
While having a robot to do our laundry might be far off, what about #ai to choose our outfits? @TheYes is building just that. Tune in Wednesday for the full episode.
This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
While having a robot to do our laundry might be far off, what about #ai to choose our outfits? @TheYes is building just that. Tune in Wednesday for the full episode.
This media couldn't be displayed. See it on the original post. -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
On this week's episode, @GustavS of @Spotify chats consumption data, Transformer neural nets, and #ml in the pursuit of listener happiness.🎶 youtu.be/dRZxUbSywSM
-
Pieter Abbeel reposted
Rowan McAllister
@rowantmc
Want to organize a workshop but unsure how? Here's a guide!
rowanmcallister.github.io/post/workshops/
From collected experiences at NeurIPs, ICML, ECCV, ICRA etc, thanks to @xinshuoweng @ermgrant @andrea_bajcsy @sociotiose @RCalandra @yaringal @pabbeel @weilunchao -
Pieter Abbeel reposted
AIX Ventures
@aixventureshq
Congrats to the entire @huggingface team on their Series C raise.
techcrunch.com/2022/05/09/hugging-face-reaches-2-billion-… -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Last week's round up of awesome robot videos by @IEEESpectrum, featuring our episode with @SkydioHQ about drones flying solo. spectrum.ieee.org/video-friday-no-time-to-dance
-
Pieter Abbeel reposted
Bianca Caban
@bianca_caban
It's inspiring to see the practical impacts on everyday life that #ai can have. Check out this podcast with @aixventureshq investment partner @pabbeel and @whisperai1 CEO @songexe where they talk about how AI is being used to improve hearing aids.
www.youtube.com/watch?v=DWXEnOKkHqg -
Pieter Abbeel
@pabbeel
This week on @therobotbrains we had @adampbry, Co-Founder/CEO of @SkydioHQ, building the most advanced AI for drones! We covered role of real and synthetic data, deep learning, consumer experience, commercial use cases, the arc of autonomy for drones.
youtu.be/DraZM4omRUk
Pieter Abbeel
@pabbeel
also on:
Google: music.youtube.com/googlepodcasts
Spotify: spoti.fi/3KNOkdX
Apple: podcasts.apple.com/us/podcast/adam-bry-of-skydio-is-leadi… -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Drones are cool but what's cooler? A drone that's so smart it flies itself getting the best footage -- while also being affordable. On today's episode, @adampbry of @SkydioHQ takes us inside his company's mission to make drones intelligent and accessible. youtu.be/DraZM4omRUk
-
Pieter Abbeel reposted
Covariant
@CovariantAI
AI-powered robotic picking is no longer just a nice to have. Examples like this @BastianSolution and @GEODIS_Group engagement prove that it has become a critical element of mainstream modern automation design.
#autostore #warehouseautomation #warehousing
roboticsandautomationnews.com/2021/10/21/geodis-and-basti… -
Pieter Abbeel reposted
Covariant
@CovariantAI
In this recent @NVIDIAGTC presentation, @MostafaRohani shares insights from our research team's work — including how it enables Covariant to unlock unprecedented ROI for our customers around the globe.
@NVIDIAI #artificialintelligence
www.youtube.com/watch?v=1QPr59OUZ5I&hss_channel=tw-106095… -
Pieter Abbeel reposted
Stephen James
@stepjamUK
New work! We envision coarse-to-fine Q-attention as a tree that can be expanded and used to accumulate value estimates across the top-k voxels at each Q-attention depth.
Allows for more robust sparse-reward, vision-based manipulation! 🧵👇
arxiv.org/abs/2204.12471
w/ @pabbeel -
Pieter Abbeel
@pabbeel
At this year's @nvidia's GTC, for the first time, we shared some specifics on the @CovariantAI Brain. Here is a video of the presentation, by our founding researcher @MostafaRohani:
www.youtube.com/watch?v=1QPr59OUZ5I
Enjoy! 🙂🎥🤓 -
Pieter Abbeel
@pabbeel
I had a unique chance to combine two of my passions: AI and sports!
@catapultsports helps athletes in many sports reach the top of their game. This year, the LA Rams winner of the 2022 Super Bowl wore Catapult Sports tech!
youtu.be/M4XuWamsVRw
Pieter Abbeel
@pabbeel
Other platforms:
Google: music.youtube.com/googlepodcasts
Spotify: spoti.fi/3kdSrp4
Apple: podcasts.apple.com/us/podcast/catapult-sports-on-ai-that-… -
Pieter Abbeel reposted
Hanie Sedghi
@HanieSedghi
We are excited to share #neurips2022 call for workshops! Please spread the word and consider submitting your proposals!
With @SungjjnAhn_ @hsuantienlin @TristanNaumann
NeurIPS Conference
@NeurIPSConf
#neurips2022 will have in-person and virtual workshops this year. Submit your proposals by June 3, 2022.
But before you do so, read about the 7 most common pitfalls of workshop proposals here
blog.neurips.cc/2022/04/24/guidance-for-neurips-workshop-…
Call for Workshops:
blog.neurips.cc/2022/04/24/neurips-2022-call-for-post-con… -
Pieter Abbeel reposted
Berkeley Computing, Data Science, and Society
@BerkeleyCDSS
🐻🎉 Congratulations to @Berkeley_EECS' @red_abebe, who has been named a 2022 @CarnegieCorp Andrew Carnegie Fellow!
This will fund her research on auditing and evaluating evidentiary statistical software used in the criminal legal system. Learn more: data.berkeley.edu/news/uc-berkeleys-rediet-abebe-named-ca… -
Pieter Abbeel reposted
Association for Computing Machinery
@TheOfficialACM
Check out the latest interview with 2021 #acmprize recipient @pabbeel 👇
Pieter Abbeel
@pabbeel
I had the honor to be interviewed by Stephen Ibaraki in context of my @TheOfficialACM Prize. Really enjoyed the interview.
Early career: youtu.be/S45cIUw1Zxg?t=55
Deep RL: youtu.be/S45cIUw1Zxg?t=399
Domain Randomization: youtu.be/S45cIUw1Zxg?t=859 -
Pieter Abbeel
@pabbeel
Had a wonderful conversation with @amitp42, co-founder of @thoughtspot, which builds natural / NLP interfaces to extract analytics from databases. Customers include Nike, Walmart, Apple.
Don't miss the part about setting aside 1hr each day for learning:
youtu.be/mwCK0Esi3NI?t=2951
Pieter Abbeel
@pabbeel
Or on your favorite podcast platform:
Spotify: spoti.fi/3uZ2Qv2
Google: music.youtube.com/googlepodcasts
Apple: podcasts.apple.com/us/podcast/amit-prakash-of-thoughtspot… -
Pieter Abbeel
@pabbeel
I had the honor to be interviewed by Stephen Ibaraki in context of my @TheOfficialACM Prize. Really enjoyed the interview.
Early career: youtu.be/S45cIUw1Zxg?t=55
Deep RL: youtu.be/S45cIUw1Zxg?t=399
Domain Randomization: youtu.be/S45cIUw1Zxg?t=859
Pieter Abbeel
@pabbeel
AGI/HER/generalization: youtu.be/S45cIUw1Zxg?t=1193
Deep Learning enough?: youtu.be/S45cIUw1Zxg?t=2007
Covariant tech philosophy: youtu.be/S45cIUw1Zxg?t=2775
Current research at Berkeley: youtu.be/S45cIUw1Zxg?t=3066
Pieter Abbeel
@pabbeel
Gradescope: youtu.be/S45cIUw1Zxg?t=3173
Covariant: youtu.be/S45cIUw1Zxg?t=3285
AIX Ventures: youtu.be/S45cIUw1Zxg?t=3622
The Robot Brains: youtu.be/S45cIUw1Zxg?t=3942 -
Pieter Abbeel
@pabbeel
Exciting announcement: in my @therobotbrains conversation with @geoffreyhinton (yes!!!!!!!!!), there will be some time for (prepared) audience questions. By Tuesday, post/like questions you have for Geoff in reply to this tweet, and I'll try to incorporate into our conversation.
Pieter Abbeel
@pabbeel
Thanks for so many great suggestions, was able to incorporate many of them! Had a gem of a conversation with Geoff on Friday. This episode should go out early June, stay tuned!
-
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Named a Visionary by @Gartner's Magic Quadrant 2x, @Thoughtspot has managed to make data querying pleasant & painless. Co-founder @amitp42 joins the pod to share how his work on Bing & @GoogleAI led to his invention. youtu.be/mwCK0Esi3NI
-
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
No NQL skills? No problem. This week's guest, @Thoughtspot is making it possible for all types of users to query their data. Full episode due out Wednesday!
-
Pieter Abbeel
@pabbeel
Re-reading #geniusmakers in preparation for the Season 2 finale guest of @therobotbrains
-
Pieter Abbeel reposted
Qi Dou
@QiDou_
How to significantly improve success rate of robotic bin-picking without need of any labelling in practice? See our new work on Self-training Sim2Real 6D Object Pose Estimation!
Thanks to great collaboration between UC Berkeley and CUHK @stepjamUK @pabbeel @KaiChen007
Stephen James
@stepjamUK
Our new Sim2Real 6D Object Pose Estimation work!
1) Train pose estimation model on sim data.
2) Use model to generate poses on unlabelled real data.
3) Auto filter generated poses and update model.
4) Repeat 2-4.
Result: SOTA performance + robot demo! 🤖
arxiv.org/abs/2204.07049 -
Pieter Abbeel reposted
Eric Horvitz
@erichorvitz
Lightning round! Pop questions from @pabbeel: youtu.be/K2bFFQj5_bA
The Robot Brains Podcast
@therobotbrains
What do Howard Pattee, @kazuoishiguro6, and ice skating all have in common? Microsoft's @erichorvitz is a fan. Check out this week's "On the DL" to learn more!
www.youtube.com/watch?v=K2bFFQj5_bA -
Pieter Abbeel reposted
Stephen James
@stepjamUK
Our new Sim2Real 6D Object Pose Estimation work!
1) Train pose estimation model on sim data.
2) Use model to generate poses on unlabelled real data.
3) Auto filter generated poses and update model.
4) Repeat 2-4.
Result: SOTA performance + robot demo! 🤖
arxiv.org/abs/2204.07049 -
Pieter Abbeel reposted
CIFAR
@CIFAR_News
Pieter Abbeel (@pabbeel @berkeley_ai), an advisor in our Learning in Machines & Brains program, penned an article emphasizing the importance of data audits in AI.
venturebeat.com/2022/04/02/the-importance-of-data-audits-… -
Pieter Abbeel reposted
Eric Horvitz
@erichorvitz
Had a blast chatting with @pabbeel on @therobotbrains. Podcast links: Apple podcasts.apple.com/us/podcast/eric-horvitz-of-microsoft-o…, Spotify spoti.fi/3jRGTrv, Google music.youtube.com/googlepodcasts @microsoft @NSF #dnn #ai @PartnershipAI #nscai @StanfordHAI @berkeley_ai @UCBerkeleyEECS @compcomcon @CovariantAI
-
Pieter Abbeel reposted
Eric Horvitz
@erichorvitz
Enjoyed the discussion with @pabbeel. Podcast can be accessed via Apple: podcasts.apple.com/us/podcast/eric-horvitz-of-microsoft-o…, Spotify: spoti.fi/3jRGTrv, and Google: music.youtube.com/googlepodcasts #ai @therobotbrains
The Robot Brains Podcast
@therobotbrains
This week we take a step back to ponder Big Tech's role & responsibility in the pursuit of building better #ai with #microsoft's @erichorvitz...tune in Wednesday for the full episode! 🙌
youtu.be/VtF1NfY-EAI -
Pieter Abbeel reposted
Covariant
@CovariantAI
This update from @KNAPP_AG_de showcases how customers like @obetaelectro, @GXOlogistics and McKeson are leveraging The Covariant Brain to automate robotic pick and place applications.
www.knapp.com/en/newsroom/press/knapp-advances-ai-robotic… -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Congrats to @DiligentRobots on their raise! @andreaT inspired us immensely during her interview on S1; she's going to change healthcare for the better. youtu.be/nUtwOUNoZw0
Diligent Robotics
@DiligentRobots
We are proud to announce our Series B raise of over $30 million! The funding will accelerate product development to meet demand for healthcare service robot & enhance interoperability, integrations & efficiency in healthcare institutions. finance.yahoo.com/news/diligent-robotics-raises-over-30-1… #seriesb
-
Pieter Abbeel reposted
Andrew Song
@songexe
Thanks for hosting me, @pabbeel! It was great to talk AI x hearing aids and the exciting future ahead on @therobotbrains!
Pieter Abbeel
@pabbeel
Hard to imagine a space more amenable to modern AI disruption than hearing aids. @WhisperAi is doing exactly that!
Co-founder/CEO @songexe is hard at work ensuring no-one has to forego being part of the conversation
youtu.be/DWXEnOKkHqg -
Pieter Abbeel
@pabbeel
Hard to imagine a space more amenable to modern AI disruption than hearing aids. @WhisperAi is doing exactly that!
Co-founder/CEO @songexe is hard at work ensuring no-one has to forego being part of the conversation
youtu.be/DWXEnOKkHqg
Pieter Abbeel
@pabbeel
Spotify: spoti.fi/3udzdFN
Apple: podcasts.apple.com/us/podcast/andrew-song-of-whisper-ai-o… -
Pieter Abbeel reposted
Patrick Collison
@patrickc
Are M1 Macs yielding a brief and golden window of highly-responsive UIs, where frameworks and libraries haven't yet had enough time to compensate with additional overhead?
-
Pieter Abbeel reposted
Lukas Biewald
@l2k
“Play is the equivalent of pre-training for RL agents.
Kids don't get scored, there's no reward or feedback, they just play. We can formalize this in our robots – that’s the RL equivalent of pre-training – it has to be reward free.” – @pabbeel
▶️ wandb.ai/wandb_fc/gradient-dissent/reports/Pieter-Abbeel-… -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Affecting 1 in 8 Americans, hearing loss is a major problem. 👂In response, @WhisperAI invented an AI-powered hearing aid that's unlike any of its predecessors.
And the best part? It keeps improving by learning on the fly & consistent software updates. www.youtube.com/watch?v=DWXEnOKkHqg -
Pieter Abbeel reposted
🅐🅜🅘🅡 🅡🅐🅗🅝🅐🅜🅐
@amirrahnama_
My favourite intro course on Reinforcement Learning is Pieter Abbeel’s “Foundations of Deep RL”: www.youtube.com/watch?v=2GwBez0D20A&list=PLwRJQ4m4UJjNymu…. Thank you, @pabbeel for uploading the videos.
-
Pieter Abbeel
@pabbeel
I'll happily sign up for that level of @therobotbrains impact if I can 😀
🍃🌳🌲 Avi Chillin ☕
@AviMohan21
@pabbeel is truly one of a kind, in that he's not only an excellent researcher, but is also heavily involved in the dissemination of knowledge about #reinforcementlearning and #robotics
Don't be surprised if, in coming years,
Association for Computing Machinery
@TheOfficialACM
Our warmest congratulations to Pieter Abbeel @pabbeel, recipient of the 2021 #acmprize for contributions to robot learning, including learning from demonstrations and deep reinforcement learning for robotic control.
Learn more about Abbeel’s work here: awards.acm.org/about/2021-acm-prize -
Pieter Abbeel
@pabbeel
I couldn't be more honored. So thankful for all my amazing collaborators that made this possible, my students and colleagues at @UCBerkeley, collaborators at @OpenAI and @CovariantAI, and of course my MS and PhD advisors @DaphneKoller and @AndrewYNg, where my AI journey started.
Association for Computing Machinery
@TheOfficialACM
Our warmest congratulations to Pieter Abbeel @pabbeel, recipient of the 2021 #acmprize for contributions to robot learning, including learning from demonstrations and deep reinforcement learning for robotic control.
Learn more about Abbeel’s work here: awards.acm.org/about/2021-acm-prize
Pieter Abbeel
@pabbeel
And thank you for all the felicitations, so nice to be part of such a wonderful community 🙏
-
Pieter Abbeel reposted
Stephen James
@stepjamUK
New! Learned Path Ranking (LPR) takes a Q-attention next-best pose, and learns to rank a set of goal-reaching paths generated by path planning, Bezier curve sampling, and a learned policy.
We can now accomplish more RLBench and real-world tasks!
w/@pabbeel
arxiv.org/abs/2204.01571 -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Hearing is one of our five major senses; it's also one of the first to go as we age.🦻 @songexe of @whisperai joins @pabbeel this week to explain the science and art behind Whisper's first-of-its-kind #artificialintelligence-powered hearing aid.
This media couldn't be displayed. See it on the original post. -
Pieter Abbeel
@pabbeel
For Ep13, so fortunate to have @OhioState Dean Ayanna Howard @robotsmarts. Worked on Mars Rover at @NASAJPL at 18, never slowed from there. Successful roboticist, entrepreneur, educator, and author of Sex, Race, Robots: How to be Human in the Age of AI
youtu.be/R40kIb3gjPI
Pieter Abbeel
@pabbeel
Spotify: spoti.fi/3wJTFzQ
Apple: podcasts.apple.com/us/podcast/dean-ayanna-howard-on-why-a…
Google: music.youtube.com/googlepodcasts -
Pieter Abbeel reposted
Stephen James
@stepjamUK
Coarse-to-fine Q-attention has been selected for an oral at #cvpr2022!🙀
The first *general* 6DoF RL-based manipulation algorithm that can train in the real world in minutes (not days/months). High-res images, and no shaped rewards!
arxiv.org/abs/2106.12534
sites.google.com/view/c2f-q-attention -
Pieter Abbeel
@pabbeel
Was fun to get to write a guest piece for @VentureBeat on what I have observed in how leading AI researchers/practitioners think about data:
venturebeat.com/2022/04/02/the-importance-of-data-audits-…
Includes insights that @karpathy, @orussakovsky, Mike Schuster shared on the podcast! -
Pieter Abbeel reposted
Ted Stinson
@TAStinson
"as AI-generated insights and predictions become the status quo, the less flashy work of data maintenance can get overlooked. It’s like building a house on sand" @pabbeel in @VentureBeat
venturebeat.com/2022/04/02/the-importance-of-data-audits-… -
Pieter Abbeel reposted
kourosh hakhamaneshi
@CyrusHakha
Can we pre-train Graph Neural Networks for few-shot circuit modeling and design?
We show that by pre-training deep GNNs to predict bias of circuits we can learn models that can be reused to do sample efficient circuit optimization and modeling.
📖 arxiv.org/abs/2203.15913
1/N -
Pieter Abbeel reposted
Alejandro Escontrela
@alescontrela
Excited to present our recent work which leverages 4.5s of mocap data to train policies that exhibit natural locomotion strategies, complete tasks, and overcome Sim2Real. All without requiring complex reward functions
Link: sites.google.com/berkeley.edu/amp-in-real/home
(1/6) -
Pieter Abbeel reposted
Jack Kelly (@jack-kelly.com on Bluesky)
@jack_kelly
I thoroughly enjoyed this podcast episode! @shakir_za talks with such enthusiasm and passion about nowcasting rainfall (and many other topics!). Highly recommended listening!
The Robot Brains Podcast
@therobotbrains
In the world of #ai, @deepmind is a massive presence.🤖 And this week's guest @shakir_za has been with them 8+ years helping lead their research while also founding @DeepIndaba & being an advocate for using #ml for positive social transformation. 🌍youtu.be/KvZGaP8qlI8
-
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Our guest @robotsmarts proves just how much good 1 person can fit into a lifetime of #robotics work. 🤖 From joining @NASAJPL at 18 to starting @GeorgiaTech HumAnS lab & then becoming Dean of @OSUengineering and also founding @Zyrobotics, she is a legend. youtu.be/R40kIb3gjPI
-
Pieter Abbeel reposted
Stephen James
@stepjamUK
New work!
1) Pre-train action-free video prediction from "wild" data (e.g. RLBench/youtube/etc).
2) Fine-tune action-conditioned video prediction to quickly learn vision-based RL in a new domain (e.g. MetaWorld).
arxiv.org/abs/2203.13880
Lead:@younggyoseo w/@kimin_le2,@pabbeel
Younggyo Seo
@younggyoseo
Can we leverage diverse out-of-domain videos for improving vision-based RL?
Yes! Excited to share APV that quickly learns world models 🌎 by fine-tuning a pre-trained action-free video prediction model.
Paper: arxiv.org/abs/2203.13880
w/ @kimin_le2 @stepjamUK @pabbeel -
Pieter Abbeel reposted
Younggyo Seo
@younggyoseo
Can we leverage diverse out-of-domain videos for improving vision-based RL?
Yes! Excited to share APV that quickly learns world models 🌎 by fine-tuning a pre-trained action-free video prediction model.
Paper: arxiv.org/abs/2203.13880
w/ @kimin_le2 @stepjamUK @pabbeel -
Pieter Abbeel reposted
KNAPP
@KNAPP_AG_de
One of the deciding factors for having a future-proof warehouse is artificial intelligence. That’s why we combine our industrial-grade Pick-it-Easy Robot with @CovariantAI. Find out more in our #partnership story. #valuechaintechpartner #robotics www.knapp.com/en/knapp-advances-ai-robotics-for-more-effi…
-
Pieter Abbeel
@pabbeel
This week we were so fortunate to have @shakir_za from @DeepMind on @therobotbrains. We talked about his 8+yrs at Deepmind, his recent Nature paper on nowcasting, AI and healthcare, AI and climate, the @DeepIndaba, his essay on queer exceptionalism.
youtu.be/KvZGaP8qlI8
Pieter Abbeel
@pabbeel
Spotify: spoti.fi/3wwGnqm
Apple: podcasts.apple.com/us/podcast/shakir-mohamed-of-deepmind-…
Google: music.youtube.com/googlepodcasts -
Pieter Abbeel reposted
Covariant
@CovariantAI
Our AI-powered Robotic Putwall will be best in show at #modex2022
Check out @therobotreport Modex preview for key event highlights, including the Covariant demo experience!
www.therobotreport.com/modex-2022-show-guide/?hss_channel… -
Pieter Abbeel reposted
ManufacturingMillennial
@MFGMillennial
In 2018 B2C eCommerce sales were $2.3 Trillion, and 𝐛𝐲 𝟐𝟎𝟐𝟒 𝐢𝐭'𝐬 𝐞𝐱𝐩𝐞𝐜𝐭𝐞𝐝 𝐭𝐨 𝐫𝐞𝐚𝐜𝐡 𝐨𝐯𝐞𝐫 $𝟔 𝐓𝐫𝐢𝐥𝐥𝐢𝐨𝐧. In order for this growth to continue, we need to automate. Learn more from @CovariantAI at Modex March 28-31!
-
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Today's "On the DL" with @shakir_za is especially enlightening with mentions of Wangari Maathai, Nelson Mandela & poetry. youtu.be/_Y3MrlZRnrw
Listen to his full interview here:
Spotify: spoti.fi/3wwGnqm
Apple: podcasts.apple.com/us/podcast/shakir-mohamed-of-deepmind-…
Google: music.youtube.com/googlepodcasts -
Pieter Abbeel reposted
AIX Ventures
@aixventureshq
We are excited to launch our $50MM Pre/Seed Fund I. Check out the AIX Ventures coverage in VentureBeat.
venturebeat.com/2022/03/24/top-ai-execs-including-richard…
@RichardSocher @shaunbjohnson @pabbeel @chrmanning @antgoldbloom @fangy23 -
Pieter Abbeel
@pabbeel
Very excited @aixventureshq officially launched today: "AI Practitioners investing in AI powered companies"
For anyone looking for jobs at amazing AI start-ups, check out the amazing opportunities at AIX portfolio companies:
careers.aixventures.com/jobs -
Pieter Abbeel reposted
Shahin Farshchi
@Farshchi
The best AI founders seek the smartest in AI: @pabbeel and @RichardSocher
Building a fund to put those founders in business and attract other amazing AI talent around them is the obvious next step
Excited to be partnered with @aixventureshq: www.wsj.com/articles/aix-ventures-gathers-top-ai-talent-t… -
Pieter Abbeel
@pabbeel
My conversation with @shakir_za was one of the most energizing and inspiring in a long time. To be released on Wednesday!
The Robot Brains Podcast
@therobotbrains
Airing this Wednesday, @shakir_za joins @pabbeel to discuss his last 8 years at @DeepMind, founding @DeepIndaba, & recent breakthroughs like using ML to solve the problem of nowcasting (short-term weather predictions ☔️)!
-
Pieter Abbeel
@pabbeel
Introducing robotics to children at a young age in the same way that we enroll them in soccer or piano lessons should be a real consideration. However the cost used to be prohibitive. But, no more, thanks to @JaredSchrieber and Revolution Robotics!
youtu.be/cYEtqZ5iDKk
Pieter Abbeel
@pabbeel
To listen to the full episode, tune in here! 🤖
Spotify: spoti.fi/34N0EfR
Apple: podcasts.apple.com/us/podcast/revolution-robotics-is-brin… -
Pieter Abbeel reposted
Marcus Frei
@MarcusFrei_
2022: A major but quiet revolution in (advanced) #robotics #ai with @pabbeel on why. www.zdnet.com/article/2022-prediction-a-major-revolution-…
-
Pieter Abbeel
@pabbeel
Loved this conversation with Julianna Fleming from @Cal_Engineer about The Robot Brains podcast!
engineering.berkeley.edu/news/2022/03/robots-ai-and-podca… -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Anyone working on a #robot to scare off wild boars? 🐗 @JaredSchrieber explains why he wants one on this week's "On the DL" youtu.be/BqCSTLxpA1U
Tune into the full episode here:
Spotify: spoti.fi/34N0EfR
Apple: podcasts.apple.com/us/podcast/revolution-robotics-is-brin…
Google: music.youtube.com/googlepodcasts -
Pieter Abbeel reposted
Covariant
@CovariantAI
Headed to #modex2022? The Covariant Brain will be on display in a variety of places, including at @ABBRobotics, Booth #b4004.
Check out this @automation_com for more.
www.automation.com/en-us/articles/february-2022/abb-advan… -
Pieter Abbeel reposted
Tech Funding News
@TFNBreakingNews
New Yorks’s Bloom grabs $1.1M to help e-commerce brands generate more sales
👉 techfundingnews.com/new-yorkss-bloom-grabs-1-1m-to-help-e…
#tech #funding #news #canada #ai #ecommercebusiness
@inovia @RichardSocher @pabbeel @chrmanning @stanford @antgoldbloom @forumventures @ystartupindex -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
For this week's episode, host @pabbeel asks, "What is the best way to endow future generations with #stem skills?"
@JaredSchrieber of non-profit, Revolution Robotics (revolutionrobotics.org), explains how #robotics teams for kids could be the key to it all. -
Pieter Abbeel
@pabbeel
Motivated by his own life experiences, @gnetscher1 is pioneering AI to help the elderly and anyone with dementia be safer in their homes.
@SafelyYou has already successfully reduced falls by 40% and ER visits resulting from falls by 80%.
YouTube: youtu.be/ndWMXgeLVwc
Pieter Abbeel
@pabbeel
Apple: podcasts.apple.com/us/podcast/george-netscher-of-safelyyo…
Spotify: spoti.fi/3tIrfTF -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
In 2015, there were 7 caregivers for every person over 80 yo.🧓By 2080? There will only be 4. Can robots fill this gap? 🤖
@SafelyYou joins the pod to share the progress of AI for fall detection, false alarm mitigation, & supporting overworked caregivers. www.youtube.com/watch?v=ndWMXgeLVwc -
Pieter Abbeel reposted
Covariant
@CovariantAI
In the latest issue of @HarvardBiz, @hjameswilson and @pauldaugh share examples of businesses that have implemented AI to solve systemic problems - including the @obetaelectro deployment of the @KNAPP_AG_de Pick-it-Easy Robot powered by Covariant!
hbr.org/2022/03/robots-need-us-more-than-we-need-them?hss… -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
What is the #1 most expensive disease in America?
Dementia.
@gnetscher1 of @SafelyYou joins the podcast this week to discuss the #ai-powered home system that can help sick ones be safer & their families spend more wisely on care. youtu.be/ffdJXrjcO_k -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Imagine an Ikea specialist digitally alongside you while building that tricky chair? With the help of AR, this is the future of customer service.
On Ep. 9, @ftasse discusses her #ai work at @streem_pro & her efforts with @black_in_ai to improve inclusion. youtu.be/kbDVKTN_4go -
Pieter Abbeel
@pabbeel
Super-inspiring conversation: @ftasse grew up in Cameroon, BS at Buea -> BS/MS at Capetown -> PhD in comp. vision Cambridge -> founder Selerio, pioneering AI for AR -> acquired by Streem, which empowers customer service to be “in” your home through AI+AR
youtu.be/kbDVKTN_4go
Pieter Abbeel
@pabbeel
Apple: podcasts.apple.com/us/podcast/flora-tasse-on-building-com…
Spotify: spoti.fi/3tguYaQ -
Pieter Abbeel reposted
Covariant
@CovariantAI
Ready to discover how the Covariant Brain is powering robots to autonomously perform millions of picks at fulfillment centers around the world?
Check out our new microsite to learn more - autonomy.covariant.ai?hss_channel=tw-1060953957248290821 -
Pieter Abbeel reposted
Alex Kendall
@alexgkendall
These questions put me on the spot! Some fun answers and I'm glad I got one back at you @pabbeel 😁
The Robot Brains Podcast
@therobotbrains
Most recent book? Mentor? Best present? Learn more about Ep. 8 guest, @alexgkendall! www.youtube.com/watch?v=36qNyLocakY
To enjoy Alex's full episode, visit the podcast platform of your choice!
Spotify: spoti.fi/3BS2XKI
Apple: podcasts.apple.com/us/podcast/how-wayve-is-teaching-cars-…
Google: music.youtube.com/googlepodcasts -
Pieter Abbeel reposted
Yuqing Du
@d_yuqing
Humans excel at generating curricula of tasks simply by interacting with other agents. Can RL agents generate similar curricula? We tackle this in our #iclr2022 paper, Multiagent Selfplay for Automatic Curriculum Generation!
w/ @pabbeel, @adityagrover_
1/8 -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Until recent years, end-to-end learning was not being built by mainstream AV companies for teaching cars to drive. @wayve_ai has changed that. 🚗
Check out our interview with @alexgkendall to learn more. www.youtube.com/watch?v=wI5MnC-WdT4 -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Last season featured experts on #autonomousvehicles including @karpathy from @Tesla, @ancadianadragan from @Waymo, & @chris_urmson from @aurora_inno. This week, @alexgkendall, the CEO of @wayve_ai, joins us to discuss his vision for AV2.0 & why end-to-end learning is the key. 🚗
-
Pieter Abbeel reposted
Łukasz Jocz
@lukaszjocz
Glad to discover @therobotbrains podcast. Thank you @pabbeel and @wightmanr for inspiring conversation. Have been using timm from its beginnings. Great example of how persistent steady effort can build fantastic stuff. youtu.be/O_w5VU72xAA
-
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
In this week's “On the DL", prolific AI/ML open source contributor @wightmanr shares why he thinks intellect is an overrated virtue. 🤔
m.youtube.com/watch?v=sKl0SIYiRzo -
Pieter Abbeel reposted
vasudev anubrolu
@vasudevanubrolu
A great podcast from @pabbeel
with @wightmanr. Insightful for anyone keen on opensource. Inspiring.
open.spotify.com/episode/2pkT4VqIIckLBEaJCNMmVB -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
A big hello to all our new followers! We're so excited to have you along for our adventures in #robotics powered by #ai & #ml. 🤖
Be sure to sign up for our newsletter for the latest and greatest from @pabbeel: tinyletter.com/TheRobotBrains -
Pieter Abbeel reposted
Frédéric Bernard
@BEAmbUSA
Great talk with @berkeley_ai professor Pieter Abbeel @pabbeel about #robotics and the impact of #artificialintelligence on our daily lives.
#dyk this 🌐 renowned robotics expert was born in Antwerp 🇧🇪? #ai #gobears @UCBerkeley @Nathalie_SF @ralphm8 -
Pieter Abbeel reposted
Angjoo Kanazawa
@akanazawa
The deadline for this summer undergraduate program is extended till 2/28!! Still have time to submit your applications!
Angjoo Kanazawa
@akanazawa
Ping to undergraduates interested in AI🌟
We're only a week away from the deadline (1/31) for the second BAIR summer undergraduate program! The idea is for HBCU and PBI students to get hands on experience in AI, it's paid & in-person (covid allowing🤞)!
Please spread the word! -
Pieter Abbeel reposted
Amrit Sahu
@AmritSa51993336
I stumbled on this video as one of regular robot brains podcast videos i watch regularly. I feel this video was a gem! @wightmanr can't really mention how much amazing video this was , and admire becoming like you. Thanks @pabbeel for such amazing content!
-
Pieter Abbeel
@pabbeel
The @wightmanr episode is now released!
The Robot Brains Podcast
@therobotbrains
A well-known contributor to @arXiv_Daily, @kaggle, @huggingface, & @github to name a few...@wightmanr joins our podcast to share what drives him to keep building better #ai & #ml models freely available for the whole world to use.🌍 www.youtube.com/watch?v=O_w5VU72xAA
-
Pieter Abbeel
@pabbeel
. @wightmanr is one of the most unique people I have met, it's kind of mind-blowing how much a single dedicated person can move the needle in an era where big companies spend billions on AI R&D
Can't wait for the episode to air later this week!
The Robot Brains Podcast
@therobotbrains
Meet @wightmanr, one of the most prolific contributors to the open source #ai community to date. His @kaggle & @github profiles speak for themselves. His full interview with @pabbeel airs this Wednesday!
-
Pieter Abbeel reposted
IEEE Awards
@IEEEAwards
Congratulations to @UCBerkeley’s Pieter Abbeel (@pabbeel) on receiving the 2022 @IEEEorg Kiyo Tomiyasu Award, sponsored by the late Dr. Kiyo Tomiyasu, @IEEE_GRSS, and @IEEEMTT, for contributions to #deeplearning for #robotics: corporate-awards.ieee.org/recipients/current-recipients/ #ieeeawards2022 #ieeetfas
-
Pieter Abbeel
@pabbeel
This week, I had to great fortune to sit down with @mitidss Professor @wucathy, sharing how machine learning will shape the transportation of the future.
YouTube: youtu.be/iMG1Fk-woX4
Pieter Abbeel
@pabbeel
Spotify: spoti.fi/337R0U3
Apple: podcasts.apple.com/us/podcast/cathy-wu-of-mit-on-the-futu…
Google: music.youtube.com/googlepodcasts -
Pieter Abbeel reposted
Hao Liu
@haoliuhl
Some thoughts on the work led by my amazing collaborators @denisyarats and @brandfonbrener.
With diverse data, many problems in RL just go away.
Bitter lesson strikes again.
ExORL arxiv.org/abs/2201.13425 could be very useful for future offline and unsupervised RL research.
Denis Yarats
@denisyarats
Currently, Offline RL data is collected under the same reward that is used for evaluation, not ideal...
@brandfonbrener and I propose an alternative approach – ExORL, that uses Unsupervised RL & relabeling to construct datasets for Offline RL.
paper: arxiv.org/abs/2201.13425
1/10 -
Pieter Abbeel reposted
Denis Yarats
@denisyarats
Currently, Offline RL data is collected under the same reward that is used for evaluation, not ideal...
@brandfonbrener and I propose an alternative approach – ExORL, that uses Unsupervised RL & relabeling to construct datasets for Offline RL.
paper: arxiv.org/abs/2201.13425
1/10 -
Pieter Abbeel reposted
Stephen James
@stepjamUK
New paper! Not all reinforcement learning problems are suited for a Gaussian policy parameterization!🤯
Plan to use 3D rotation or 6D pose as part of an action space? Consider the Bingham distribution! 🧵1/5
Paper: arxiv.org/abs/2202.03957
Code: github.com/stepjam/BPP
w/@pabbeel -
Pieter Abbeel reposted
Tanay Tandon
@tanay_tandon
had a nice time chatting with prof @pabbeel about @athelas, and why we're excited about ML in healthcare.
Pieter Abbeel
@pabbeel
I had the great fortune to sit down with AI/healthcare visionary @tanay_tandon, Founder/CEO of @athelas, making blood diagnostics more affordable & efficient.
Athelas just announced a 1.5BB valuation from its recent fundraising round.
youtu.be/eMMnzLoryYE -
Pieter Abbeel
@pabbeel
I had the great fortune to sit down with AI/healthcare visionary @tanay_tandon, Founder/CEO of @athelas, making blood diagnostics more affordable & efficient.
Athelas just announced a 1.5BB valuation from its recent fundraising round.
youtu.be/eMMnzLoryYE
Pieter Abbeel
@pabbeel
Watch at the youtube link above, or listen here:
Google: music.youtube.com/googlepodcasts
Spotify: spoti.fi/3HpGhTR
Apple: podcasts.apple.com/us/podcast/tanay-tandon-of-athelas-on-… -
Pieter Abbeel reposted
Misha Laskin
@MishaLaskin
New paper on unsupervised skill discovery - Contrastive Intrinsic Control.
Tl;dr exploration with contrastive skill learning substantially improves prior skill discovery methods (by 1.8x)! Achieves leading unsupervised RL results.
arxiv.org/abs/2202.00161
Learn more 👇
1/N -
Pieter Abbeel reposted
Covariant
@CovariantAI
Interested to learn how @KNAPP_AG_de views the current state of warehouse automation, and where we're headed? Spoiler, it includes our AI Robotics platform!
Check out this recent interview by @Warehouse_News.
warehousenews.co.uk/2021/12/automation-knapp-uk-the-value… -
Pieter Abbeel
@pabbeel
So excited to (very soon) release this episode. @tanay_tandon is one the clearest thinkers and most driven executors I have met. Not to mention such an important mission, to make healthcare accessible, affordable, and better, starting with at home blood tests that work!
The Robot Brains Podcast
@therobotbrains
On the heels of @Athelas announcing its $1.5B valuation, our episode with @tanay_tandon airs this Wednesday. Tune in to learn why investors like @sequoia and @generalcatalyst are betting on this #machinelearning-powered #healthcare start-up.
-
Pieter Abbeel
@pabbeel
Had a really great time sitting down with @Benedictevans, a long-time analyst who goes very deep into trends in AI and tech more generally. He also spent time at @a16z. Come find out Ben’s observed trends and predictions!

www.youtube.com/watch?v=M8UL_2hj60o
Pieter Abbeel
@pabbeel
. @Benedictevans also has an amazing newsletter, here: www.ben-evans.com/newsletter
Listen to the full episode here:
Google: music.youtube.com/googlepodcasts
Spotify: spoti.fi/3fVvu81
Apple: podcasts.apple.com/us/podcast/what-matters-in-tech-accord… -
Pieter Abbeel reposted
CIFAR
@CIFAR_News
Listen: Canada CIFAR AI Chair @david_rolnick (@Mila_Quebec @mcgillu) is interviewed by CIFAR advisor @pabbeel about his groundbreaking work using machine learning in the fight against climate change.
open.spotify.com/episode/1m7V84DdplFczUD7mLgRdW -
Pieter Abbeel reposted
Deepak Pathak
@deepakpathak
LLMs like GPT-3 and Codex contain rich world knowledge. In this fun study, we ask if GPT like models can plan actions for embodied agents.
Turns out, with apt sanity checks, even vanilla LLMs without any finetuning can generate good high-level plans given a low-level controller. -
Pieter Abbeel
@pabbeel
I had an absolutely wonderful conversation with @david_rolnick on the role ML could play towards mitigating climate change, links below!
watch: youtu.be/84IIXsOl6No
Spotify: spoti.fi/3GNaxry
Apple: podcasts.apple.com/us/podcast/david-rolnick-on-how-machin…
Google: music.youtube.com/googlepodcasts
The Robot Brains Podcast
@therobotbrains
Climate change is one of the biggest threats to our world's future.🌍 Meet @david_rolnick, a scientist campaigning for the broader use of #ai & #ml for research like resource depletion prediction, hyperlocal weather simulations, and ecological monitoring. youtu.be/84IIXsOl6No
-
Pieter Abbeel reposted
utku
@utkuevci
"...How could such an obvious thing was not known (a large nn works). Well, people were really focused on ML models where they can prove that there is an algorithm with perfect solution. But whenever you put this constraint... you end up restricting the power of your model." Ilya
-
Pieter Abbeel
@pabbeel
Did you know @Etsy has over 100 million unique products?
In Ep2 S2, CTO @MikeFisher_Fish, former Army pilot, explains how AI is helping buyers and sellers find each other on Etsy, and more generally the ever growing role of AI at Etsy.
youtu.be/0WJltc6j50k
Pieter Abbeel
@pabbeel
And the audio streams:
Spotify: spoti.fi/3zRQe9D
Apple: podcasts.apple.com/us/podcast/mike-fisher-of-etsy-talks-a…
Google: music.youtube.com/googlepodcasts -
Pieter Abbeel reposted
Covariant
@CovariantAI
Interested to learn how @Etsy approaches AI and computer vision? In the latest episode of @therobotbrains, @pabbeel interviews Etsy CTO, @MikeFisher_Fish
Watch it here: www.youtube.com/watch?v=0WJltc6j50k&hss_channel=tw-106095… or check it out on your favorite podcasting platform! -
Pieter Abbeel reposted
Anthony Goldbloom
@antgoldbloom
Started listening to @pabbeel's @therobotbrains's Podcast: www.therobotbrains.ai. Pieter is a great interviewer and highly recommend for anyone interested in learning more about the big breakthroughs and hearing from key thought leaders in AI/ML.
-
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Join us as @Etsy CTO @MikeFisher_Fish dives into the #ecommerce site's digital transformation over the past 3 years. He discusses everything from his early days as a pilot in the Army to the technical details of Etsy's varied #ai innovations. youtu.be/0WJltc6j50k
-
Pieter Abbeel reposted
Covariant
@CovariantAI
2021 was quite the year for AI robotics. While the number of milestones is too long to list here, we’re especially proud of our AI Robotic Putwall (seen below) winning a product of the year award from @modernmhmag and @mathandling247.
Happy new year! -
Pieter Abbeel reposted
Alice Patel
@aliceypatel
On the DL…like on the down low but it also stands for deep learning. These puns don’t write themselves, guys.
The Robot Brains Podcast
@therobotbrains
In case you missed it, @svlevine joined our podcast to discuss his recent #airobotics research...
@pabbeel also used the time to ask him 10 v important questions for our new segment, "On the DL"! www.youtube.com/watch?v=yivTPnFA4N0 -
Pieter Abbeel
@pabbeel
First episode of season 2 is out, with guest @svlevine!!
Spotify: spoti.fi/3zqFnU5
Apple: podcasts.apple.com/us/podcast/sergey-levine-explains-the-…
YouTube: www.youtube.com/watch?v=vMEdchIjzfE -
Pieter Abbeel
@pabbeel
congratulations, and much looking forward to the next generation of developer tools for robotics!
Adrian Macneil
@adrianmacneil
Excited to share our seed funding! Thanks to our amazing investors @AmplifyPartners @BoostVC @PathbreakerVC @pabbeel @kvogt @karrisaarinen @FEhrsam @lindaxie @dwr
Read more about our vision: foxglove.dev/blog/foxglove-raises-seed-funding-3.7M -
Pieter Abbeel
@pabbeel
So AMAZING @Astro_Woody, congratulations!!!! Never met someone so dedicated. You told me at PhD visit days how every single thing you do is all part of your aspiration/plan to become an astronaut. And here you are, selected for @NASA / @SpaceX mission!!!
www.space.com/nasa-astronauts-spacex-crew-6-mission -
Pieter Abbeel reposted
The Robot Brains Podcast
@therobotbrains
Season 2 starts in January 2022! Are you subscribed yet?
With guests like @svlevine, @robotsmarts, &
@gustavs you'll learn about the latest and greatest in #ai & #robotics including never-before-heard stories about the people who build them. 🤖 youtu.be/NzMcR6jPg14 -
Pieter Abbeel
@pabbeel
Was wonderful to be hosted on @thenewstack and discuss the differences between traditional pre-programmed-motion robots (as seen in car manufacturing lines) and the new generation of AI-powered robots helping in warehouses
The New Stack
@thenewstack
AI-Controlled Robots Need to Be Smarter thenewstack.io/ai-controlled-robots-need-to-be-smarter/ @bcamerongain @alexwilliams @pabbeel @CovariantAI #ai #machinelearning #robotics
-
Pieter Abbeel reposted
Misha Laskin
@MishaLaskin
Over the last few years, unsupervised learning has produced breakthroughs in CV and NLP. Will the same thing happen in RL? @denisyarats and I wrote a blog post discussing unsupervised vs supervised RL and the unsupervised RL benchmark.
bair.berkeley.edu/blog/2021/12/15/unsupervised-rl/ -
Pieter Abbeel reposted
Denis Yarats
@denisyarats
@MishaLaskin wrote a nice blog post about our recent work on establishing benchmarking for Unsupervised RL, please take a look: bair.berkeley.edu/blog/2021/12/15/unsupervised-rl/
-
Pieter Abbeel
@pabbeel
What a wonderful initiative! The amount of time most scientists spend on fundraising / reporting / reviewing proposals / bending their research agenda to funding agency goals is mindblowing. Here is a chance for biomedical researchers to actually spend their time on science!
Patrick Collison
@patrickc
I've long been interested in new ways to organize science and enable curiosity-driven discovery. Today, in partnership with @Stanford, @UCBerkeley, and @UCSF, we're excited to announce Arc Institute, a new undertaking in this vein: arcinstitute.org.
Pieter Abbeel
@pabbeel
alright, I now better get back my focus back onto some proposal writing... (sadly not kidding)
-
Pieter Abbeel reposted
Ted Stinson
@TAStinson
“there will be a day soon in which the majority of the items in our household were touched by a robot… before reaching us”
@pabbeel interviewed ahead of season 2 of @therobotbrains release
ZDNET
@ZDNET
2022: A major revolution in robotics www.zdnet.com/article/2022-prediction-a-major-revolution-…
-
Pieter Abbeel reposted
Spiros Margaris
@SpirosMargaris
2022:
A major revolution in #robotics
zd.net/3m3ldKk #fintech #ai #machinelearning #deeplearning #artificialintelligence @ZDNet @gcnichs @Paula_Piccard @jblefevre60 @pierrepinna @ipfconline1 @HeinzVHoenen @Nicochan33 @Fabriziobustama @darioandriani @RoboSrgryInvest -
Pieter Abbeel
@pabbeel
Kenji Doya reminded us of the neuroscience version of @ylecun 's LeCake at this year's Deep RL workshop.
neurips.cc/virtual/2021/workshop/21848 -
Pieter Abbeel
@pabbeel
Curious how Robinson Crusoe could make an appearance in a Deep RL Talk?
--> Check out @svlevine's Opinion on RL Talk here: neurips.cc/virtual/2021/workshop/21848! -
Pieter Abbeel reposted
Covariant
@CovariantAI
Today we’re excited to announce that @modernmhmag and @mathandling247 have named the Covariant AI Robotic Putwall, Readers Choice Product of the Year!
A huge thanks to our entire team and our partners and customers around the globe!
#materialshandling
www.mmh.com/article/mhpn_mh_247_readers_choice_products_o… -
Pieter Abbeel reposted
UC Berkeley EECS
@Berkeley_EECS
New Kavli Center at UC Berkeley to be led by Stuart Russell. news.berkeley.edu/2021/12/09/new-kavli-center-at-uc-berke…
-
Pieter Abbeel
@pabbeel
Excited to share the epic Season 1 finale of @therobotbrains with the amazing @ilyasut Co-Founder/Chief Scientist @OpenAI!
His breakthroughs include AlexNet, seq2seq, MT, GPT, CLIP, DallE,Codex.
"According to my parents, I've been talking about AI at a relatively early age" 😂
Pieter Abbeel
@pabbeel
. @ilyasut shares so many gems on his thinking behind his works.
He also talks about seeking out @geoffreyhinton once in Canada.
Spotify: spoti.fi/3ylBcXf
Apple: podcasts.apple.com/gb/podcast/ilya-sutskever-explains-the…
Google: music.youtube.com/googlepodcasts
PS: 250,000+ citations < 10 years out of PhD 🤯🤯 -
Pieter Abbeel
@pabbeel
Excited to share a 6-lecture series on Foundations of Deep RL:
youtube.com/playlist?list=PLwRJQ4m4UJjNymuBM9RdmB3Z9N5-0IlY0
MDP basics, value & policy iteration, max-ent, DQN, policy gradient, TRPO, PPO, DDPG, SAC, model-based RL.
And went in rabbit hole of fixing captions, way too much time, but enjoy the read 😅 -
Pieter Abbeel
@pabbeel
On past @therobotbrains episodes, we discussed how AI and robots learn. In Ep11, we are mixing it up, discussing how children learn w/ the amazing @AlisonGopnik. Alison has pioneered our understanding of how children learn, with many lessons for AI.
spoti.fi/3pEn212 -
Pieter Abbeel
@pabbeel
Very excited to share the launch of my new podcast, The Robot Brains. The first episode was just released. My first guest is brilliant AI researcher, educator, and Director of AI and Autopilot Vision at Tesla @karpathy!
www.therobotbrains.ai -
Pieter Abbeel
@pabbeel
Been wanting to learn about CPC, CPC-v2, MoCo, SimCLR, MoCo-v2, and other self-supervised learning techniques?
Wonderful lecture by @Aravind7694 for the @UCBerkeley Deep Unsupervised Learning class:
youtu.be/dMUes74-nYY -
Pieter Abbeel
@pabbeel
One of my favorites from most recent offering of CS287 Advanced Robotics?
Exam study handout summarizing all the main math in ~20pp. Incl. MaxEnt RL, CEM, LQR, Penalty Method, RRTs, Particle Filters, Policy Gradient, TRPO, PPO, Q-learning, DDPG, SAC,
people.eecs.berkeley.edu/~pabbeel/cs287-fa19/exam/cs287-f… -
Pieter Abbeel
@pabbeel
Much looking forward to catching up with friends and colleagues, and making new ones at @CMU_Robotics tomorrow!
Seminar at 3pm in 1305 Newell Simon Hall.
www.ri.cmu.edu/event/ri-seminar-pieter-abbeel-uc-berkeley… -
Pieter Abbeel
@pabbeel
The website for the #neurips2019 Deep RL Workshop is now live!
Paper submissions due September 9th.
Co-organized with @jachiam0, @chelseabfinn
Joelle Pineau, David Silver, Satinder Singh, Carlos Florensa, Chris Grimm, @hrtang_alex, @vivek_veeriah
sites.google.com/view/deep-rl-workshop-neurips-2019/ -
Pieter Abbeel
@pabbeel
All materials of @berkeley_ai Deep Unsupervised Learning now up:
sites.google.com/view/berkeley-cs294-158-sp19/home
Great semester w/@peterxichen,@Aravind7694,@hojonathanho, and guest inst. @AlecRad,@ilyasut,A Efros,@avdnoord
Covers: AR / PixelCNN, Flow models, VAE, GAN, self-supervised learning, etc... -
Pieter Abbeel
@pabbeel
Thrilled to be teaching a new course on Deep Unsupervised Learning with @peterxichen (ImprovedGAN, InfoGAN, PixelCNN++, VLAE, PixelSNAIL, Flow++), @hojonathanho (Flow++, GAIL), @aravind7694 (Flow++).
Follow along here:
sites.google.com/view/berkeley-cs294-158-sp19/home
Lecture vids go up once captioned -
Pieter Abbeel
@pabbeel
Great reading list on Safety and Control for AGI:
inst.eecs.berkeley.edu/~cs294-149/fa18/
taught by Andrew Critch and Stuart Russell from @berkeley_ai -
Pieter Abbeel
@pabbeel
The Robot of the Year Award aims at supporting the development of the most promising robots and AI, for the benefit of human society and aligned with well-defined ethical criteria in a number of key industrial sectors. Consider applying!
www.therobotoftheyear.com/ourjury/ -
Pieter Abbeel
@pabbeel
Excited to share I'm joining AI@TheHouse as Co-Founder and Faculty Partner along with an amazing group of my colleagues from UC Berkeley. We're building a new engine for AI innovation that closely couples academia, startups, industry, and education. medium.com/@thehousecal/announcing-ai-the-house-990e77857228
-
Pieter Abbeel
@pabbeel
Video of my keynote on Deep Learning and Robotics is now available here: www.facebook.com/nipsfoundation/videos/1554594181298482/
(includes lots of meta-learning :) . #nips2017 -
Pieter Abbeel
@pabbeel
Ever wanted to show a robot what to do, and have the robot learn from that? @chelseabfinn and team are hosting live demos showcasing one-shot imitation and video prediction at NIPS tonight! Starting at 6:30pm, at D6.
-
Pieter Abbeel
@pabbeel
Deep RL Bootcamp lecture videos and labs release sites.google.com/view/deep-rl-bootcamp/lectures! :) [Organized by @pabbeel, Peter Chen, Rocky Duan, @karpathy]
-
Pieter Abbeel
@pabbeel
Beautiful explanation of Max-Ent Deep RL by Haoran Tang and Tuomas Haarnoja! Also model-free RL (from scratch) on real robot :)
Berkeley AI Research
@berkeley_ai
Blog post:"Learning Diverse Skills via Maximum Entropy Deep Reinforcement Learning" by Haoran Tang & Tuomas Haarnoja bair.berkeley.edu/blog/2017/10/06/soft-q-learning/
-
Pieter Abbeel
@pabbeel
Excellent lecture on model based RL at DeepRLBootcamp by @chelseabfinn
-
Pieter Abbeel
@pabbeel
Robot: (1) watch, (2) similar task, (3) hallucinate success, (4) RL to success. arxiv.org/abs/1707.03374 by Y Liu*,A Gupta*,P Abbeel,S Levine
-
Pieter Abbeel
@pabbeel
Berkeley AI Research (BAIR) is launching an official blog, with weekly posts on cutting-edge AI research results!
bair.berkeley.edu/blog/










































































































































































