examples of what I mean
ncbi.nlm.nih.gov/pmc/articles/P…
I still haven't heard a good answer to this question, on or off the podcast. AI researchers often tell me, "Don't worry bout it, scale solves this." But what is the rebuttal to someone who argues that this indicates a fundamental limitation?- Full episode, for those interested:Here is my conversation with Dario Amodei, CEO of Anthropic. We discuss: - why human level AI is 2-3 years away - race dynamics with OpenAI & China - $10 billion training runs, bioterrorism, alignment, cyberattacks, scaling...
- examples of what I mean ncbi.nlm.nih.gov/pmc/articles/P…
- Best answer so far:Replying to @dwarkesh_sptl;dr: Maybe learning simple things (basic knowledge, heuristics, etc) actually lowers the loss more than learning sophisticated things (algorithms associated with higher cognition that we really care about), and the sophisticated things will eventually be learned as scaling
These are prediction engines, which is why they can appear to think but also simultaneously don't think like us. A toddler can point out a bird in the sky, but the machine has to be trained to see it, even if it can tell that the colours are a bit different.
