Are LLMs going to eventually become the architecture that powers self-driving cars?
Seeing them play Portal and other video games, I'm curious if they will eventually help solve that last N% of self-driving.
LLM's are a funny technology because on the one hand this is all undeniably impressive at the rate of what's changed from them, and yet despite that I find myself disappointed by the lack of breakthroughs for things I don't find interesting. I like math and programming, and LLM's are pretty good at it, when are they going to get good at folding laundry for me? I think a lot of robotics work promises to solve this category of "boring" breakthroughs, and I'm optimistic we'll be able to achieve it, i just wonder when
The bottleneck is not really the intelligence here.
We can build robots that do the things you want.
We can't ship them because they break all the time with current technology. It would be a tough sell to have to being in a 100kg robot for servicing every few weeks.
This was cars in the first several decades of automobiles. And here we are.
Pretty soon. Sunday Robotics had a 3 hour stream of folding clothes with 99% accuracy. You can watch it for yourself. There’s a lot of “hand” companies with very compelling videos just over the last two months. Then there was Figure’s multi day livestream of package manipulation that was very impressive. Physical LLMs are definitely coming. Given enough training data we know LLMs can output coherent data in any space, it’s just a matter of time.
The limitations that they state shouldn't really make much of a impact. But it was nice of them (and not to mention, real unbiased research) to mention those. Kudos to them!
I enjoy hearing that they were ran on medium effort.
GPT-6 is multimodal, LLMs alone have no vision capability