It's unfortunate that there's so little (none in the article, just 1 comment here as of this writing) mention of the Turing Test. The whole premise of the paper that introduced that was that "do machines think" is such a hard question to define that you have to frame the question differently. And it's ironic that we seem to talk about the Turing Test less than ever now that systems almost everyone can access can arguably pass it now.
Provocative title with a much more reasoned lede.
I'm pretty sure a set of workshops isn't ACTUALLY going to solve a problem that philosophers have been at each other's throats for for the past half century.
But BOY does it get people talking!
Both sides of the debate have capital-O Opinions, and how else did you want to drum up interest for a set of mathematics workshops. O:-)
Yes, that word 'think' is a hell of a distraction. We've replaced that bit of the title with a less provocative / more interesting phrase from the article.
Hopefully we can talk about the actual math and stuff (although the article doesn't go into much of that).
artificial intelligence in place of artificial knowledge is confusing distraction too, at least artificial is a good reminder since beginning
(IMHO its not provocative but well catching a point.. about so called "intelligence" - what if we could look for intelligent knowledge - made in not statistic but semantic(?) and converging way - instead - of being distracted, afraid and.. outpriced, peanuts for the essence it doesn't have ?)
[dead]
I wish I would've learned about ANNs in elementary school. It looks like a worthwhile and cool lesson package, if only they'd do away with the idiotic dogma...
I took an AI class in 2010 and even then was mostly taught that neural networks aren't very useful and you should use random forests and MCMC.
My other memory of this class is the professor spending a lecture telling us he'd just been promoted to full professor and how important that was, and the last time I looked him up, he'd become a director at Google Cloud.
People slag off kids using LLMs but they have the advantage that you can ask them about cutting edge stuff and get good answers. Elementary school teachers on the other hand probably generally aren't that up on machine learning.
Don't think this is very good - more of a report of their activities. Underdelivers on the headline.
I think we all intuitively knew this but it's pretty cool.
I think there's a huge divide between the type of people on HN and everyone else, in whether you might know this intuitively or not. Intuition is based on the sum of your experiences.
[stub for offtopicness]
(in this case, thinkiness)
I love the idea of educating students on the math behind AI to demystify them. But I think it's a little weird to assert "AI is not magic and AI systems do not think. It’s just maths." Equivalent statements could be made about how human brains are not magic, just biology - yet I think we still think.
I agree saying "they don't think" and leaving it at that isn't particularly useful or insightful, it's like saying "submarines don't swim" and refusing to elaborate further. It can be useful if you extend it to "they don't think like you do". Concepts like finite context windows, or the fact that the model is "frozen" and stateless, or the idea that you can transfer conversations between models are trivial if you know a bit about how LLMs work, but extremely baffling otherwise.
> Concepts like
> finite context windows
like a human has
> or the fact that the model is "frozen" and stateless,
much like a human adult. Models get updated at a slower frequency than humans. AI systems have access to fetch new information and store it for context.
> or the idea that you can transfer conversations between models are trivial
because computers are better-organized than humanity.
My context window is about a day. I can remember what I had for lunch today, and sometimes what I had for lunch yesterday. Beyond that, my lunches are gone from my context window and are only in my training data. I have vague ideas about what dishes I ate, but don't remember what days specifically. If I had to tell you what separate dishes I ate in the same meal, I don't have specific memories of that. I remember I ate fried plantains, and I ate beans & rice. I assume they were on the same day because they are from the same cuisine, and am confident enough that I would bet money on it, but I don't know for certain.
One of my earliest memories is of painting a ceramic mug when I was about 3 years old. The only reason I remember it is because every now and then I think about what my earliest memory is, and then I refresh my memory of it. I used to remember a few other things from when I was slightly older, but no longer do, because I haven't had reasons to think of them.
I don't think humans have specific black and white differences between types of knowledge that way LLMs do, but there is definitely a lot of behavior that is similar to context window vs training data (and a gradient in between). We remember recent things a lot better than less recent things. The quantity of stuff we can remember in our "working memory" is approximately finite. If you try to hold a complex thought in your mind, you can probably do that indefinitely, but if you then try to hold a second equally complex thought as well, you'll often lose the details of the first thought and need to reread or rederive those details.
Wouldn't context be comparable to human short term memory, which could be neurons firing in a certain pattern repeatedly to keep it there?
How would you say human short term memory works if not by repeated firing (similar to repeatedly putting same tokens in over and over)?
A lot of people genuinely can't remember what they did an hour ago, but to be very clear you're implying that an LLM can't "remember" something from an hour, or three hours ago, when it's the opposite.
I can restart a conversation with an LLM 15 days later and the state is exactly as it was.
Can't do that with a human.
The idea that humans have a longer, more stable context window than LLM's, CAN or is even LIKELY to be true given certain activities but please let's be honest about this.
If you talk to someone for an hour about a technical conversation I would guesstimate that 90% of humans would immediately start to lose track of details in about 10 minutes. So they write things down, or they mentally repeat things to themselves they know or have recognized they keep forgetting.
I know this because it's happened continually in tech companies decade after decade.
LLM's have already passed the Turing test. They continue to pass it. They fool and outsmart people day after day.
I'm no fan of the hype AI is receiving, especially around overstating its impact in technical domains, but pretending that LLM's can't or don't consistently perform better than most human adults on a variety of different activities is complete non-sense.
I do hope you're able to remember what buttons you just pressed without staring at your hands while doing so to keep it in your working memory
I do hope you're able to remember what was your browser tab 5 tab switches ago without keeping track of it...
> much like a human adult.
it doesn't sound like you really understand what these statements mean. if LLMs are like any humans it's those with late stage dementia, not healthy adults
It's just provencial nonsense, there's no sound reasoning to it. Reductionism being taken and used as a form of refutation is a pretty common cargo culting behavior I've found.
Overwhelmingly, I just don't think the majority of human beings have the mental toolset to work with ambiguous philosophical contexts. They'll still try though, and what you get out of that is a 4th order baudrillardian simulation of reason.
"Just" is used here as a reductive device. You reduce others to a few sentences.
Sentences constructed of words and representations of ideas defined long before you existed. I question whether you can work with ambiguous contexts as you have had the privilege of them being laid out in language for you already by the time you were born.
From my reference frame you appear to merely be circumlocuting from memory, and become the argument you make about others.
Thinking is undefined so all statements about it are unverifiable.
I would say a different problem:
There's many definitions of "thinking".
AI and brains can do some, AI and brains definitely provably cannot do others, some others are untestable at present, and nobody really knows enough about what human brains do to be able to tell if or when some existing or future AI can do whatever is needed for the stuff we find special about ourselves.
A lot of people use different definitions, and respond to anyone pointing this out by denying the issue and claiming their own definition is the only sensible one and "obviously" everyone else (who isn't a weird pedant) uses it.
This is not a meta-question.
The definition of "thinking" in any of the parent comments or TFA is actually not defined. Like literally no statements are made about what is being tested.
So, if we had that we could actually discuss it. Otherwise it's just opinions about what a person believes thinking is, combined with what LLMs are doing + what the person believes they themselves do + what they believe others do. It's entirely subjective with very low SNR b/c of those confounding factors.
Halting problem.
There are people who insist that the halting problem "proves" that machines will never be able to think. That this means they don't understand the difference between writing down (or generating a proof of) the halting problem and the implications of the halting problem, does not stop them from using it.
Computing the Kolmorgorov constant?
Statements like "it is bound by the laws of physics" are not "verifiable" by your definition, and yet we safely assume it is true of everything. Everything except the human brain, that is, for which wild speculation that it may be supernatural is seemingly considered rational discussion so long as it satisfies people's needs to believe that they are somehow special in the universe.
It is bound by the same laws of physics as everything else, so no, not special.
What has it got to do with anything whether brains function like computers? This is only relevant if you define thinking as something only the brain can do, and then nothing that doesn't work like a brain can think. This would be like defining flight as "what birds do" and then saying airplanes can't fly because they don't work like birds.
And then what do you even mean by "a computer?" This falls into the same trap because it sounds like your statement that brains don't function like a computer is really saying "brains don't function like the computers I am familiar with." But this would be like saying quantum computers aren't computers because they don't work like classical computers.
They are, apparently, proscribed by the totality of the laws of physics. For now.
But every unicorn is bound by the laws of physics.
Do you think that thinking is undefinable ? If thinking is definable, then all statements about it aren't unverifiable.
Yes, that's a problem of me not being a native english speaker. "All x aren't y" may mean "not all x are y" in my tongue. Not a single x is y is more what we would say in the previous case. But in our case we would say there are x that aren't y.
If thinking is definable, it is wrong that all statements about it are unverifiable (i.e. there are statements about it that are verifiable.)
Well, basic shit.
Is this some self refuting sentence?
I think they meant "Cannot evaluate : (is <undefined> like x ?), argument missing"
edit : Thinking is undefined, statements about undefined cannot be verified.
is a meta-level grammar the same as an object-level grammar?
Is reasoning undefined? That's what usually meant by "thinking".
At the end of the day most people would agree that if something is able to solve a problem without a lookup table / memorisation that it used reasoning to reach the answer. You are really just splitting hairs here.
The difference between thinking and reasoning is that I can "think" that Elvis is still alive, Jewish space lasers are responsible for California wildfires, and Trump was re-elected president in 2020, but I cannot "reason" myself into those positions.
It ties into another aspect of these perennial threads, where it is somehow OK for humans to engage in deluded or hallucinatory thought, but when an AI model does it, it proves they don't "think."
>Equivalent statements could be made about how human brains are not magic, just biology - yet I think we still think.
They're not equivalent at all because the AI is by no means biological. "It's just maths" could maybe be applied to humans but this is backed entirely by supposition and would ultimately just be an assumption of its own conclusion - that human brains work on the same underlying principles as AI because it is assumed that they're based on the same underlying principles as AI.
Unless you're supposing something mystical or supernatural about how brains work, then yes, it is "just" math, there is nothing else it could be. All of the evidence we have shows it's an electrochemical network of neurons processing information. There's no evidence that suggests anything different, or even the need for anything different. There's no missing piece or deep mystery to it.
It's on those who want alternative explanations to demonstrate even the slightest need for them exists - there is no scientific evidence that exists which suggests the operation of brains as computers, as information processors, as substrate independent equivalents to Turing machines, are insufficient to any of the cognitive phenomena known across the entire domain of human knowledge.
We are brains in bone vats, connected to a wonderful and sophisticated sensorimotor platform, and our brains create the reality we experience by processing sensor data and constructing a simulation which we perceive as subjective experience.
The explanation we have is sufficient to the phenomenon. There's no need or benefit for searching for unnecessarily complicated alternative interpretations.
If you aren't satisfied with the explanation, it doesn't really matter - to quote one of Neil DeGrasse Tyson's best turns of phrase: "the universe is under no obligation to make sense to you"
If you can find evidence, any evidence whatsoever, and that evidence withstands scientific scrutiny, and it demands more than the explanation we currently have, then by all means, chase it down and find out more about how cognition works and expand our understanding of the universe. It simply doesn't look like we need anything more, in principle, to fully explain the nature of biological intelligence, and consciousness, and how brains work.
Mind as interdimensional radios, mystical souls and spirits, quantum tubules, none of that stuff has any basis in a ruthlessly rational and scientific review of the science of cognition.
That doesn't preclude souls and supernatural appearing phenomena or all manner of "other" things happening. There's simply no need to tie it in with cognition - neurotransmitters, biological networks, electrical activity, that's all you need.
>it doesn't really matter - to quote one of Neil DeGrasse Tyson's best turns of phrase: "the universe is under no obligation to make sense to you"
Right back at you, brochacho. I'm not the one making a positive claim here. You're the one who insists that it must work in a specific way because you can't conceive of any alternatives. I have never seen ANY evidence or study linking any existent AI or computer system to human cognition.
>There's no need or benefit for searching for unnecessarily complicated alternative interpretations.
Thanks, if it's alright with you I might borrow this argument next time somebody tries to tell me the world isn't flat.
>It simply doesn't look
That's one of those phrases you use when you're REALLY confident that you know what you're talking about.
> like we need anything more, in principle, to fully explain the nature of biological intelligence, and consciousness, and how brains work.
Please fully explain the nature of intelligence, consciousness, and how brains work.
>Mind as interdimensional radios, mystical souls and spirits, quantum tubules, none of that stuff has any basis in a ruthlessly rational and scientific review of the science of cognition.
well i definitely never said anything even remotely similar to that. If i didn't know any better i might call this argument a "hallucination".
AI operates alot like trees do as they are both using maths under the hood.
This is the point, we don't know the delta between brains and AI any assumption is equivalent to my statement.
Math is a superset of both processes (can model/implement both), but that doesn't imply that they are equivalent.
Well, a better retort would be "Human brains are not magic, just physics. Protons, neutrons and electrons don't think".
But I think most people get what GP means.
Panpsychism is actually quite reasonable in part because it changes the questions you ask. Instead of “does it think” you need to ask “in what ways can it think, and in what ways is it constrained? What types of ‘experience/qualia’ can this system have, and what can’t it have?”
When you think in these terms, it becomes clear that LLMs can’t have certain types of experiences (eg see in color) but could have others.
A “weak” panpsychism approach would just stop at ruling out experience or qualia based on physical limitations. Yet I prefer the “strong” pansychist theory that whatever is not forbidden is required, which begins to get really interesting (would imply that for example an LLM actually experiences the interaction you have with it, in some way).
But parent didn't try to apply "it's just maths" to humans. He said one could just as easily say, as some do: "Humans are just biology, hence they're not magic". Our understanding of mathematics, including the maths of transformer models is limited, just as our understanding of biology. Some behaviors of these models have taken researches by surprise, and future surprises are not at all excluded. We don't know exactly how far they will evolve.
As for applying the word thinking to AI systems, it's already in common usage and this won't change. We don't have any other candidate words, and this one is the closest existing word for referencing a computational process which, one must admit, is in many ways (but definitely not in all ways) analogous to human thought.
Human brains might not be explained by the same type of math AI is explained with, but it will be some kind of math...
Human brains and experiences seem to be constrained by the laws of quantum physics, which can be simulated to arbitrary fidelity on a computer. Nit sure where Godel’s incompleteness theory would even come in here…
Brain damage? If thought was outside physics, it would be a bit more durable than Humpty Dumpty.
Please explain, because this interpetation of "Godel" is highly nonstandard.
> that human brains work on the same underlying principles as AI
That wasn't the assumption though, it was only that human brains work by some "non-magical" electro-chemical process which could be described as a mechanism, whether that mechanism followed the same principles of AI or not.
Straw man. The person who you're responding to talked about "equivalent statements" (emphasis added), whereas you appear to be talking about equivalent objects (AIs vs. brains), and pointing out the obvious flaw in this argument, that AIs aren't biology. The obvious flaw in the wrong argument, that is.
Yeah. This whole AI situation has really exposed how bad most people are at considering the ontological and semantic content of the words they use.
A large number of adults I encounter are functionally illiterate, including business people in very high up positions. They are also almost 100% MATHEMATICALLY illiterate, not only unable to solve basic algebra and geometry problems, but completely unable to reason about statistical and probabilistic situations encountered in every day life. This is why gambling is so popular and why people are constantly fooled by politicians. It's bad enough to be without words in the modern world, but being without numbers makes you vulnerable to all manner of manipulations.
Gambling exists more because of people dopamine systems than math...though I get the overall drift. People are fooled by politicians because ?? Also not really math related I think.
Indeed, people confidently assert as established fact things like "brains are bound by the laws of physics" and therefore "there can't be anything special" about them, so "consciousness is an illusion" and "the mind is a computer", all with absolute conviction but with very little understanding of what physics and maths really do and do not say about the universe. It's a quasi-religious faith in a thing not fully comprehended. I hope in the long run some humility in the face of reality will eventually be (re)learned.
My "position" isn't remotely that. The problem with "brains are bound by the laws of physics" isn't that there's something special about brains. It's that physics doesn't consist of "laws" that things are "bound" by. It consists of theories that attempt to describe.
These theories are enormously successful, but they are also known to be variously incomplete, inconsistent, non-deterministic, philosophically problematic, open to multiple interpretations and only partially understood in their implications, with links between descriptions of things at different scales a particularly challenging and little understood topic. The more you learn about physics (and while I'm no physicist, I have a degree in the subject and have learned a great deal more since) the more you understand the limits of what we know.
Anybody who thinks there's no mystery to physics just doesn't know much about it. Anybody who confidently asserts as fact things like "the brain consists of protons, neutrons and electrons so it's impossible for it to do anything a computer can't do" is deducing things from their own ignorance.
This. People do not understand the implications of the most basic facts of modern science. Gravitation is instantaneous action at a distance via an "occult" force (to quote Newton's contemporaries).
Lot's of assumptions about humanity and how unique we are constantly get paraded in this conversation. Ironically, the people who tout those perspectives are the least likely to understand why we're really not all that "special" from a very factual and academic perspective.
You'd think it would unlock certain concepts for this class of people, but ironically, they seem unable to digest the information and update their context.
I have yet to hear any plausible definition of "thought" that convincingly places LLMs and brains on opposite sides of it without being obviously contrived for that purpose.
Define "think".
We observe through our senses geometric relationships.
Syntax is exactly that; letters, sentences, paragraphs organized in spatial/geometric relationships.
At best thought is recreation of neural networks in the brain which only exist as spatial relationships.
Our senses operate on spatial relationships; enough light to work by, and food relative to stomach to satisfy our biological impetus to survive (which is spatial relationships of biochemistry).
The idea of "thought" as anything but biology makes little sense to me then as a root source is clearly observable. Humanity, roughly, repeats the same social story. All that thought does not seem to be all that useful as we end up in the same place; the majority as serfs of aristocracy.
Personally would prefer less "thought" role-play and more people taking the load of the labor they exploit to enable them to sit and "think".
A college level approach could look at the line between Math/Science/Physics and Philosophy. One thing from the article that stood out to me was that the introduction to their approach started with a problem about classifying a traffic light. Is it red or green?
But the accompanying XY plot showed samples that overlapped or at least were ambiguous. I immediately lost a lot of my interest in their approach, because traffic lights by design are very clearly red, or green. There aren't mauve or taupe lights that the local populace laughs at and says, "yes, that's mostly red."
I like the idea of studying math by using ML examples. I'm guessing this is a first step and future education will have better examples to learn from.
> traffic lights by design are very clearly red, or green
I suspect you feel this because you are observing the output of a very sophisticated image processing pipeline in your own head. When you are dealing with raw matrixes of rgb values it all becomes a lot more fuzzy. Especially when you encounter different illuminations, exposures and the cropping of the traffic light has noise on it. Not saying it is some intractably hard machine vision problem, because it is not. But there is some variety and fuzzyness there in the raw sensor measurements.
The human mind is not just biology in the same way that LLMs are just math.
AI systems compute and humans think. One is math and the other biology.
But they are two different things with overlapping qualities.
It's like MDMA and falling in love. They have many overlapping quantities but no one would claim one is the other.
There's a huge amount of money going to convincing people that AI is magic or better than people. The reprogramming is necessary.
That's where these threads always end up. Someone asserts, almost violently, that AI does not and/or cannot "think." When asked how to falsify their assertion, perhaps by explaining what exactly is unique about the human brain that cannot and/or will not be possible to emulate, that's the last anyone ever hears from them. At least until the next "AI can't think" story gets posted.
The same arguments that appeared in 2015 inevitably get trotted out, almost verbatim, ten years later. It would be amusing on other sites, but it's just pathetic here.
Consider that you might have become polarized yourself. I often encounter good arguments against current AI systems emulating all essential aspects of human thinking. For example, the fact that they can't learn from few examples, that they can't perform simple mathematical operations without access to external help (via tool calling) or that they have to expend so much more energy to do their magic (and yes, to me they are a bit magical), which makes some wonder if what these models do is a form of refined brute-force search, rather than ideating.
Personally, I'm ok with reusing the word "thinking", but there are dogmatic stances on both sides. For example, lots of people decreeing that biology in the end can't but reduce to maths, since "what else could it be". The truth is we don't actually know if it is possible, for any conceivable computational system, to emulate all essential aspects of human thought. There are good arguments for this (in)possibility, like those presented by Roger Penrose in "the Emperor's new Mind" and "Shadows of the Mind".
> For one thing, yes, they can
That's post-training. The complaint I'm referring to is to the huge amounts of data (end energy) required during training - which is also a form of learning, after all. Sure, there are counter-arguments, for example pointing to the huge amount of non-textual data a child ingests, but these counter-arguments are not waterproof themselves (for example, one can point out that we are discussing text-only tasks). The discussion can go on and on, my point was only that cogent arguments are indeed often presented, which you were denying above.
> there are plenty of humans who seemingly cannot
This particular defense of LLMs has always puzzled me. By this measure, simply because there are sufficiently impaired humans, AGI has already been achieved many decades ago.
> But yet you are fine with humans requiring a calculator to perform similar tasks
I'm talking about tasks like multiplying two 4-digit numbers (let's say 8-digit, just to be safe, for reasoning models), which 5th or 6th graders in the US are expected to be able to do with no problem - without using a calculator.
> To me, the magical thing is that they have shown us, more or less undeniably (Penrose notwithstanding), that that is all we do.
Or, to put it more tersely, they have shown you that that is all we do. Penrose, myself, and lots of others don't see it quite like that. (Feeling quite comfortable being classed in the same camp with the greatest living physicist, honestly. ;) To me what LLMs do is approximate one aspect of our minds. But I have a strong hunch that the rabbit hole goes much deeper, your assessment notwithstanding.
Usually it is the work of the one claiming something to prove it. So if you believe that AI does "think" you are expected to show me that it really does. Claiming it "thinks - prove otherwise" is just bad form and also opens the discussion up for moving the goalposts just as you did with your brain emulation statement. Or you could just not accept any argument made or circumvent it by stating the one trying to disprove your assertion got the definition wrong. There are countless ways to start a bad faith argument using this methodology, hence: Define property -> prove property.
Conversely, if the one asserting something doesn't want to define it there is no useful conversation to be had. (as in: AI doesn't think - I won't tell you what I mean by think)
PS: Asking someone to falsify their own assertion doesn't seem a good strategy here.
PPS: Even if everything about the human brain can be emulated, that does not constitute progress for your argument, since now you'd have to assert that AI emulates the human brain perfectly before it is complete. There is no direct connection between "This AI does not think" to "The human brain can be fully emulated". Also the difference between "does not" and "can not" is big enough here that mangling them together is inappropriate.
It definietly includes myself, I don't have the interest to stay updated here.
For that matter I have no opinion on if AI does think or not, I simply don't care. Therefore I also really can't answer your question in what more a model has to do to establish that they are thinking (does being able to use all major forms of reasoning constitute the capability of thought to you?). I can say however, that any such proof would have to be on a case-by-case basis given my current understanding on AI is designed.
Well first of all I never claimed that I was capable of thinking (smirk). We also haven't agreed on a definition of "thinking" yet, so as you can read in my previous comment, there is no meaningful conversation to be had. I also don't understand how your oddly aggresive phrasing adds to the conversation, but if it helps you: my rights and protections do not depend on whether I'm able to prove to you that I am thinking. (It also derails the conversation for what it's worth - it's a good strategy in the debating club, but these are about winning or loosing and not about fostering and obtaining knowledge)
Whatever you meant to say with "Sometimes, because of the consequences of otherwise, the order gets reversed" eludes me as well.
Someone asserts, almost religiously, that LLMs do and/or can "think." When asked how to falsify their assertion, perhaps by explaining what exactly is "thinking" in the human brain that can and/or will be possible to emulate...
This entire debate over the years is because so many confidently assert that AI can think, or that AI will soon be our God, or our ruler, etc.
One mostly sees people aggressively claiming they can’t, ever. On the other side people seem to simply allow that they might, or might eventually.
But the Turing test (which I concede, LLMs do pass) doesn't test if some system is thinking; it tests if the system can convince an unbiased observer that it is thinking. I cannot come up with a better "is this thing thinking" test, but that doesn't mean that such a test can't exist; I'm sure there are much smarter people then me trying to solve this problem.
A couple decades of chess programs nods knowingly: "First time?"
Computers can perform math and numerous other tasks billions of times faster than humans, whats your point?
This is exactly the problem. Claims about AI are unfalsifiable, thus your various non-sequiturs about AI 'thinking'.
god of the gaps
We don't know how brains work.
We really don't know how consciousness works. The popular theories that it's emergent might be proven correct, or might be proven to be like the idea that phlogiston built up in a vacuum, putting out flames.
I feel like these conversations really miss the mark: whether an LLM thinks or not is not a relevant question. It is a bit like asking “what color is an Xray?” or “what does the number 7 taste like?”
The reason I say this is because an LLM is not a complete self-contained thing if you want to compare it to a human being. It is a building block. Your brain thinks. Your prefrontal cortex however is not a complete system and if you somehow managed to extract it and wire it up to a serial terminal I suspect you’d be pretty disappointed in what it would be capable of on its own.
I want to be clear that I am not making an argument that once we hook up sensory inputs and motion outputs as well as motivations, fears, anxieties, desires, pain and pleasure centers, memory systems, sense of time, balance, fatigue, etc. to an LLM that we would get a thinking feeling conscious being. I suspect it would take something more sophisticated than an LLM. But my point is that even if an LLM was that building block, I don’t think the question of whether it is capable of thought is the right question.
> The reason I say this is because an LLM is not a complete self-contained thing if you want to compare it to a human being.
The AI companies themselves are the ones drawing the parallels to a human being. Look at how any of these LLM products are marketed and described.
Is it not within our capacity on HN to ignore whatever the marketers say and speak to the underlying technology?
There was around 500 comments on the OpenAI + Disney story, so the evidence points to "no".
they are discussing it?
Why do people call is "Artificial Intelligence" when it could be called "Statistical Model for Choosing Data"?
"Intelligence" implies "thinking" for most people, just as "Learning" in machine learning implies "understanding" for most people. The algorithms created neither 'think' nor 'understand' and until you understand that, it may be difficult to accurately judge the value of the results produced by these systems.
If we say “artificial flavoring”, we have a sense that it is an emulation of something real, and often a poor one.
Why, when we use the term for AI, do we skip over this distinction and expect it to be as good as the original—- or better?
That wouldn’t be artificial intelligence, it would just be the original artifact: “intelligence”.
Actually I think the name is apt. It's artificial. It's like how an "artificial banana" isn't actually a banana. It doesn't have to be real thinking or real learning, it just has to look intelligent (which it does).
The term was coined in 1955 to describe "learning or any other feature of intelligence" simulated by a machine [1]. The same proposal does list using natural language as one of the aspects of "the artificial intelligence problem"
It's not a perfect term, but we have been using it for seven full decades to include all of machine learning and plenty of things even less intelligent
1: https://www-formal.stanford.edu/jmc/history/dartmouth/dartmo...
How do you feel about Business Intelligence as a term?
Like the old saying:
Military justice is to justice as military music is to music
“what does the number 7 taste like?” is a nonsense question.
"how much thinking time did the LLMs get when getting gold in the maths olympiad" is not a nonsense question. Four and a half hours apparently. Different thing.
You could go on to ask if saying humans thinking about the problem is thinking but LLMs thinking about the problem is not thinking and if so why? Maybe only synapses count?
The part that's most infuriating is that we don't have to speculate at all. Any discussion or philosophizing beyond the literal computer science is simply misinformation.
There's absolutely no similarity between what computer hardware does and what a brain does. People will twist and stretch things and tickle the imagination of the naive layperson and that's just wrong. We seriously have to cut this out already.
Anthropomorphizing is dangerous even for other topics, and long understood to be before computers came around. Why do we allow this?
The way we talk about computer science today sounds about as ridiculous as invoking magic or deities to explain what we now consider high school physics or chemistry. I am aware that the future usually sees the past as primitive, but why can't we try to seem less dumb at least this time around?
> There's absolutely no similarity between what computer hardware does and what a brain does.
But at very least there's also no similarity between what computer hardware does and what even the simplest of LLMs do. They don't run on eg. x86_64 , else qemu would be sufficient for inferencing.
Similarity of the hardware is absolutely irrelevant when we're talking about emergent behavior like "thought".
You should take your complaints to OpenAI, who constantly write like LLMs think in the exact same sense as humans; here a random example:
> Large language models (LLMs) can be dishonest when reporting on their actions and beliefs -- for example, they may overstate their confidence in factual claims or cover up evidence of covert actions
They have a product to sell based on the idea AGI is right around the corner. You can’t trust Sam Altman as far as you can throw him.
Still, the sales pitch has worked to unlock huge liquidity for him so there’s that.
Still making predictions is a big part of what brains do though not the only thing. Someone wise said that LLM intelligence is a new kind of intelligence, like how animal intelligence is different from ours but is still intelligence but needs to be characterized to understand differences.
> Someone wise said that LLM intelligence is a new kind of intelligence
So long as you accept the slide ruler as a "new kind of intelligence" everything will probably work out fine, it's the Altmannian insistence that only the LLM is of the new kind that is silly.
This is the underlying problem behind syncophantcy.
I saw a YouTube video about a investigative youtuber Eddy Burback who very easily convinced chat4 that he should cut off all contact with friends and family, move to a cabin in the desert, eat baby food, wrap himself in alfoil, etc just feeding his own (faked) mistakes and delusions. "What you are doing is important, trust your instincts".
Wven if AI could hypothetically be 100x as smart as a human under the hood, it still doesn't care. It doesn't do what it thinks it should, it doesn't do what it needs to do, it does what we train it to.
We train in humanities weaknesses and follies. AI can hypothetically exceed humanity in some respects, but in other respects it is a very hard to control power tool.
AI is optimised, and optimised functions always "hack" the evaluation function. In the case of AI, the evaluation function includes human flaws. AI is trained to tell us what we want to hear.
Elon Musk sees the problem, but his solution is to try to make it think more like him, and even if that succeeds it just magnifies his own weaknesses.
Has anyone read the book criticising Ray Dalio? He is a very successful hedge fund manager, who decided that he could solve the problem of finding a replacement by psychology evaluation and training people to think like him. But even his smartest employees didn't think like him, they just (reading between the lines) gamed his system. Their incentives weren't his incentives - he could demand radical honesty and integrity but that doesn't work so well when he would (of course) reward the people who agreed with him, rather than the people who would tell him he was screwing up. His organisation (apparently) became a bunch of even more radical syncopants due to his efforts to weed out syncophantcy.
You can replicate all calculations done by LLMs with pen and paper. It would take ages to calculate anything, but it's possible. I don't think that pen and paper will ever "think", regardless of how complex the calculations involved are.
The official name is https://en.wikipedia.org/wiki/Chinese_room
The opinions are exactly the same than about LLM.
And the counter argument is also exactly the same. Imagine you take one neuron from a brain and replace it with an artificial piece of electronics (e.g. some transistors) that only generates specific outputs based on inputs, exactly like the neuron does. Now replace another neuron. And another. Eventually, you will have the entire brain replaced with a huge set of fundamentally super simple transistors. I.e. a computer. If you believe that consciousness or the ability to think disappears somewhere during this process, then you are essentially believing in some religious meta-physics or soul-like component in our brains that can not be measured. But if it can not be measured, it fundamentally can not affect you in any way. So it doesn't matter for the experiment in the end, because the outcome would be exactly the same. The only reason you might think that you are conscious and the computer is not is because you believe so. But to an outsider observer, belief is all it is. Basically religion.
There are indeed many people trying to justify this magical thinking by seeking something, anything in the brain that is out of the ordinary. They've been unsuccessful so far.
Penrose comes to mind, he will die on the hill that the brain involves quantum computations somehow, to explain his dualist position of "the soul being the entity responsible for deciding how the quantum states within the brain collapse, hence somehow controlling the body" (I am grossly simplifying). But even if that was the case, if the brain did involve quantum computations, those are still, well, computable. They just involve some amount of randomness, but so what? To continue with grandparent's experiment, you'd have to replace biological neurons with tiny quantum computer neurons instead, but the gist is the same.
> The complete function may or may not be pen and paper-able.
Would you mind expanding on this? At a base read, it seems you implying magic exists.
> component in our brains that can not be measured.
"Can not be measured", probably not. "We don't know how to measure", almost certainly.
I am capable of belief, and I've seen no evidence that the computer is. It's also possible that I'm the only person that is conscious. It's even possible that you are!
A: X, because Y
B: But Y would also imply Z
C: A was never arguing for Z! This is a strawman!
I don’t understand why people say the Chinese Room thing would prove LLMs don’t think, to me it’s obvious that the person doesn’t understand Chinese but the process does, similarly the CPU itself doesn’t understand the concepts an LLM can work with but the LLM itself does, or a neuron doesn’t understand concepts but the entire structure of your brain does
The concept of understanding emerges on a higher level from the way the neurons (biological or virtual) are connected, or the way the instructions being followed by the human in the Chinese room process the information
But really this is a philosophical/definitional thing about what you call “thinking”
Edit: I see my take on this is listed on the page as the “System reply”
If 100 top-notch philosophers disagree with you, that means you get 100 citations from top-notch philosophers. :-P
Check out eg Dennett.... or ... his opionions about Searle; Have fun with eg... this:
"By Searle’s own count, there are over a hundred published attacks on it. He can count them, but I guess he can’t read them, for in all those years he has never to my knowledge responded in detail to the dozens of devastating criticisms they contain;"
https://www.nybooks.com/articles/1995/12/21/the-mystery-of-c...
I don't see the relevance of that argument (which other responders to your post have pointed out as Searle's Chinese Room argument). The pen and paper are of course not doing any thinking, but then the pen isn't doing any writing on its own, either. It's the system of pen + paper + human that's doing the thinking.
The idea of my argument is that I notice that people project some "ethereal" properties over computations that happen in the... computer. Probably because electricity is involved, making things show up as "magic" from our point of view, making it easier to project consciousness or thinking onto the device. The cloud makes that even more abstract. But if you are aware that the transistors are just a medium that replicates what we already did for ages with knots, fingers, and paint, it gets easier to see them as plain objects. Even the resulting artifacts that the machine produces are only something meaningful from our point of view, because you need prior knowledge to read the output signals. So yeah, those devices end up being an extension of ourselves.
"'where is the consciousness' in such a system": One could ask the same of humans: where is the consciousness? The modern answer is (somewhere) in the brain, and I admit that's likely true. But we have no proof--no evidence, really--that our consciousness is not in some other dimension, and our brains could be receiving different kinds of signals from our souls in that other dimension, like TV sets receive audio and video signals from an old fashioned broadcast TV station.
If you put a droplet of water in a warm bowl every 12 hours, the bowl will remain empty as the water will evaporate. That does not mean that if you put a trillion droplets in every twelve hours it will still remain empty.
It will also not be empty if I put the bowl in the sea, which is a remark about the nature of thoughthat that proves exactly as much as your comment.
The point I was trying to make was that the time you use to perform the calculation may change whether there is an "experience" on behalf of the calculation. Without specifying the basis if subjectivity, you can't rule anything out as far as what matters and what doesn't. Maybe the speed or locality with which the calculations happen matters. Like the water drops: given the same amount of time, eventually all the water will evaporate in either case leading to the same end state, but the the intermediate states are very different.
You can replicate the entire universe with pen and paper (or a bunch of rocks). It would take an unimaginably long time, and we haven't discovered all the calculations you'd need to do yet, but presumably they exist and this could be done.
Does that actually make a universe? I don't know!
The comic is meant to be a joke, I think, but I find myself thinking about it all the time!!!
Even worse, as we are part of the universe, we would need to simulate ourselves and the very simulation that we are creating. You would also need to replicate the simulation of the simulation, leading to an eternal loop that would demand infinite matter and time (and would still not be enough!). Probably, you can't simulate something while being part of it.
It doesn’t need to be our universe, just a universe.
The question is, are the people in the simulated universe real people? Do they think and feel like we do—are they conscious? Either answer seems like it can’t possibly be right!
You're arguing against Functionalism [0], of which I'd encourage you to at least read the Wikipedia page. Why would doing the brain's computations on pen and paper rather than on wetware lead to different outcomes? And how?
Connect your pen and paper operator to a brainless human body, and you got something indistinguishable from a regular alive human.
[0] https://en.wikipedia.org/wiki/Functionalism_%28philosophy_of...
You can simulate a human brain on pen and paper too.
> You can simulate a human brain on pen and paper too.
That's an assumption, though. A plausible assumption, but still an assumption.
We know you can execute an LLM on pen and paper, because people built them and they're understood well enough that we could list the calculations you'd need to do. We don't know enough about the human brain to create a similar list, so I don't think you can reasonably make a stronger statement than "you could probably simulate..." without getting ahead of yourself.
There are some quantum effects in the brain (for some people, that's a possible source of consciousness). We can simulate quantum effects, but here comes the tricky part: even if our simulation matches the probability, say 70/30 of something happening, what guarantees that our simulation would take the same path as the object being simulated?
Consider for example Orch OR theory. If it or something like it were to be accurate, the brain would not "obey physical laws that can be described by mathematics".
This is basically the Church-Turing thesis and one of the motivations of using tape(paper) and an arbitrary alphabet in the Turing machine model.
It's been kinda discussed to oblivion in the last century, interesting that it seems people don't realize the "existing literature" and repeat the same arguments (not saying anyone is wrong).
The simulation isn't an operating brain. It's a description of one. What it "means" is imposed by us, what it actually is, is a shitload of graphite marks on paper or relays flipping around or rocks on sand or (pick your medium).
An arbitrarily-perfect simulation of a burning candle will never, ever melt wax.
An LLM is always a description. An LLM operating on a computer is identical to a description of it operating on paper (if much faster).
They do have a valid subtle point though.
If we don't think the candle in a simulated universe is a "real candle", why do we consider the intelligence in a simulated universe possibly "real intelligence"?
Being a functionalist ( https://en.wikipedia.org/wiki/Functionalism_(philosophy_of_m... ) myself, I don't know the answer on the top of my head.
I believe that the important part of a brain is the computation it's carrying out. I would call this computation thinking and say it's responsible for consciousness. I think we agree that this computation would be identical if it were simulated on a computer or paper. If you pushed me on what exactly it means for a computation to physically happen and create consciousness, I would have to move to statements I'd call dubious conjectures rather than beliefs - your points in other threads about relying on interpretation have made me think more carefully about this.
Thanks for stating your views clearly. I have some questions to try and understand them better:
Would you say you're sure that you aren't in a simulation while acknowledging that a simulated version of you would say the same?
What do you think happens to someone whose neurons get replaced by small computers one by one (if you're happy to assume for the sake of argument that such a thing is possible without changing the person's behavior)?
It seems to me that the distinction becomes irrelevant as soon as you connect inputs and outputs to the real world. You wouldn't say that a 737 autopilot can never, ever fly a real jet and yet it behaves exactly the same whether it's up in the sky or hooked up to recorded/simulated signals on a test bench.
Here is a thought experiment:
Build a simulation of creatures that evolve from simple structures (think RNA, DNA).
Now, if in this simulation, after many many iterations, the creatures start talking about consciousness, what does that tell us?
> An arbitrarily-perfect simulation of a burning candle will never, ever melt wax.
It might if the simulation includes humans observing the candle.
It's an open problem whether you can or not.
Our brains actually do something, may be the difference. They're a thing happening, not a description of a thing happening.
Whatever that something that it actually does in the real, physical world is produces the cogito in cogito, ergo sum and I doubt you can get it just by describing what all the subatomic particles are doing, any more than a computer or pen-and-paper simulated hurricane can knock your house down, no matter how perfectly simulated.
So the brain is a mathematical artifact that operates independently from time? It just happens to be implemented using physics? Somehow I doubt it.
The brain follows the laws of physics. The laws of physics can be closely approximated by mathematical models. Thus, the brain can be closely approximated by mathematical models.
Parent said replicate, as in deterministically
Wouldn't 'thinking' need to be updating the model of reality (LLM is not yet that, just words) - at every step doing again all that extensive calculations as when/to creating/approximating that/better model (learning) ?
Expecting machines to think is.. like magical thinking (but they are good at calculations indeed).
I wish we didn't use the word intelligence in context of LLMs - shortly there is Essence and the rest.. is only slope - into all possible combinations of Markov Chains - may they have sense or not I don't see how part of some calculation could recognize it, or that to be possible from inside (of calculation, that doesn't even consider that).
Aside of artificial knowledge (out of senses, experience, context lengths.. - confabulating but not knowing that), I wish to see an intelligent knowledge - made in kind of semantic way - allowed to expand using not yet obvious (but existing - not random) connections. I wouldn't expect it to think (humans think, digitals calculate). But I would expect it to have a tendency to be coming closer (not further) in reflecting/modeling reality and expanding implications.
Thinking is different than forming long term memories.
An LLM could be thinking in one of two ways. Either between adding each individual token, or collectively across multiple tokens. At the individual token level the physical mechanism doesn’t seem to fit the definition being essentially reflexive action, but across multiple tokens that’s a little more questionable especially as multiple approaches are used.
An LLM ..is calculated ..from language (or from things being said by humans before being true or not). It's not some antropomorfic process what using the word thinking would suggest (to sell well).
> across multiple tokens
- but how many ? how many of them happen in sole person life ? How many in some calculation ? Does it matter, if a calculation doesn't reflect it but stay all the same ? (conversation with.. a radio - would it have any sense ?)
That's right when saying selecting not calculating a chess move - assuming you are outside of Plato's cave (Popper).
But now, I see this: the truth is static and non-profit, but calculating something can be sold again and again, if you have a hammer (processing) everything looks like a nail, to sell well the word thinking had to be used instead of excuse for every time results being different (like the shadows) - then, we can have only things that let someone else keep making profits: JS, LLM, whatever.. (just not.. "XSLT" alike).
(yet, I need to study for your second sentence;)
> to calculate a route
(In math class I've been told that not some number but the formula is the expected result..
.. then people are now asking: "Just show me the prompt instead !" and.. done. ;) )
- how to prove that humans can argue endlessly like an llm?
- ragebait them by saying AIs don’t think
- …
LLMs don't really think, they emulate their training data. Which has a lot of examples of humans walking through problems to arrive at an answer. So naturally, if we prompt an LLM to do the same, it will emulate those examples (which tend to be more correct).
LLMs are BAD at evaluating earlier thinking errors, precisely because there's not copious examples of text where humans thinking through a problem, screwing up, going back, correcting their earlier statement, and continuing. (a good example catches these and corrects them)
Claude code is actually great at that
Given that the headline is:
> Secondary school maths showing that AI systems don’t think
And the article contains the quotes:
> the team wants to tackle a major and common misconception: that students think that ANN systems learn, recognise, see, and understand, when really it’s all just maths.
> The team is taking very complex ideas and reducing them to such an extent that we can use secondary classroom maths to show that AI is not magic and AI systems do not think.
This is not off topic
If it comes to the correct answer I don't particularly care how it got there.
In most cases, you don’t know if it came to the correct answer.
In every reasonable use case for LLMs verifying the answer is trivial. Does the code do what I wanted it to? Does it link to a source that corroborates the response?
If you're asking for things you can't easily verify you're barking up the wrong tree.
How do you know if it came to the right answer?
It's not always the case, but often verifying an answer is far easier than coming up with the answer in the first place. That's precisely the principle behind the RSA algorithm for cryptography.
Sure, it's easy to check ((sqrt(x-3)+1)/(x/8)) is less than 4. Now do it without calculus.
Very much like this effect https://www.reddit.com/r/opticalillusions/comments/1cedtcp/s... . Shouldn't hide complexity under a truth value.
A lot of the drama here is due to the ambiguity of what the word 'think' is supposed to mean. One camp associates 'thinking' to consciousness, another does not. I personally believe it is possible to create an animal-like or human-like intelligence, without consciousness existing in the system. I personally would still describe whatever processing that system is doing as 'thinking'. Others believe in "substrate independence"; they think any such system must be consciousness.
(Sneaking a bit of belief in here, to me "substrate independence" is a more extreme position than the idea that a system could be made which is intelligent but not conscious, hence I find it implausible.)
@dang offtopicness started from using word thinking in place of calculating what is the common objection in this thread.
Yes. But it's offtopic because the presence of a provocative word 'thinking' in the title led to a lot of generic tangents that don't engage with anything interesting in the article, and mostly just express people's pre-existing associations about a controversial point.
Trying to avoid this kind of thing is why the guidelines say things like:
"Eschew flamebait. Avoid generic tangents."
"Please don't pick the most provocative thing in an article or post to complain about in the thread. Find something interesting to respond to instead."
This article doesn’t really show anything near what the title assets.
> the team wants to tackle a major and common misconception: that students think that ANN systems learn, recognise, see, and understand, when really it’s all just maths
This is completely idiotic. Do these people actually believe that showing it can't be actual thought because it is described by math?
Wait until they hear about the physics/maths related to neurons firing!
<think>Ok, the user is claiming that... </think> ....
[flagged]
Maybe so, but please don't post unsubstantive comments to Hacker News.
Do we think?
By every scientific measure we have the answer is no. It’s just electrical current taking the path of least resistance through connected neurons mixed with cell death.
The fact a human brain peaks at IQ around 200 is fascinating. Can the scale even go higher? It would seem no since nothing has achieved a higher score it must not exist.
The IQ scale is constantly adjusted to keep the peak of the curve at 100 and the standard deviation around 15. To say it peaks around 200 is a pretty gross misunderstanding of what IQ means.
3 years ago this is the kind of posts that end up in /r/im14andthisisdeep
> “The question of whether a computer can think is no more interesting than the question of whether a submarine can swim.” ~ Edsger W. Dijkstra
The point of the Turing Test is that if there is no extrinsic difference between a human and a machine the intrinsic difference is moot for practical purposes. That is not an argument to whether a machine (with linear algebra, machine learning, large language models, or any other method) can think or what constitutes thinking or consciousness.
The Chinese Room thought experiment is a compliment on the intrinsic side of the comparison: https://en.wikipedia.org/wiki/Chinese_room
I kind of agree but I think the point is what people mean by words is vague, so he said:
>Instead of attempting such a definition I shall replace the question by another, which is closely related to it and is expressed in relatively unambiguous words.
which is can you tell the AI answers from the humans ones in a test. It then becomes an experimental result rather than what you mean by 'think' or maybe by 'extrinsic difference'.
The Chinese Room is a pretty useless thought exercise I think. It's an example which if you believe machines can't think seems like an utterly obvious result, and if you believe machines can think it's just obviously wrong.
People used to take it surprisingly seriously. Now it's hard to make the argument that machines can't understand say Chinese when you can give a Chinese document to a machine and ask it questions about it and get pretty good answers.
s/compliment/complement/
Good luck.
Look it up. Words have meanings.
>And it's ironic that we seem to talk about the Turing Test less than ever now that systems almost everyone can access can arguably pass it now.
Has everyone hastily agreed that it has been passed? Do people argue that a human can't figure out it's talking to an LLM if the user is aware that LLMs exist in the world and is aware of their limitations and that the chat log is able to extend to infinity ( "infinity" is a proxy here for any sufficient time, it could be minutes, days, months, or years)?
In fact, it is blindly easy for these systems to fail the Turing test at the moment. No human would have the patience to continue a conversation indefinitely without telling the person on the other side to kindly fuck off.
No, they haven't agreed because there was never a practical definition of the test. Turing had a game:
>It is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart front the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either "X is A and Y is B" or "X is B and Y is A." The interrogator is allowed to put questions to A and B.
>We now ask the question, "What will happen when a machine takes the part of A in this game?" Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman?
(some bits removed)
It was done more as thought experiment. As a practical test it would probably be too easy to fake with ELIZA type programs to be a good test. So computers could probably pass but it's not really hard enough for most people's idea of AI.
The definition seems to suffice if you give the interrogator as much time as they want and don't limit their world knowledge, which the definition that you cited doesn't seem to limit or constrain? By "world knowledge" I mean any knowledge that includes and is not limited to knowledge about how the machine works and its limitations. Therefore if the machine can't fool Alan Turing specifically then it fails even though it might have fooled some random Joe who's been living under a rock.
Hence since current LLMs are bound to hallucinate given enough time and seem not to able to maintain a conversation context window as robustly as humans, they would inevitably fail?
I’d argue the goalposts have moved substantially over the past decade. The LLMs we casually use in ChatGPT today would have been described as AGI by many people 15, 10, maybe even 5 years ago.