Is Artificial General Intelligence Here?
“For the first time in human history, we are no longer alone in the space of general intelligence.”
[image made with ChatGPT]
In the piece, Eddy Keming Chen (Philosophy, UCSD), Mikhail Belkin (Computer Science, UCSD), Leon Bergen (Linguistics, UCSD), and David Danks (Philosophy, UVA) offer what they think is a reasonable clarification of the idea of general intelligence (AGI), evidence that today’s AI fits with this clarified definition, address an array of objections, and discuss why this is important. You can read the article here.
They acknowledge that when it comes to general intelligence, “there is no ‘bright line’ test for its presence—any exact threshold is inevitably arbitrary”. Still, progress in figuring out what has general intelligence is possible, since, “we recognize paradigm cases without needing exact boundaries.” For example, “humans are paradigm examples of general intelligence; a pocket calculator lacks it, despite superhuman ability at calculations.”
In assessing what has general intelligence, we need to make sure we’re not demanding more for non-humans than for humans, they think. So when it comes to methodology, they say: “When we assess general intelligence or ability in other humans, we do not attempt to peer inside their heads to verify understanding—we infer it from behaviour, conversation and problem-solving. No single test is definitive, but evidence accumulates. The same applies to artificial systems.”
They then survey some of that evidence, observing that some current AI’s perform not just at “Turing-test level” proficiency but in ways comparable to expert humans. They note that “current LLMs even exceed what we demand of humans: we credit individual people with general intelligence on the basis of much weaker evidence.”
They also respond to various objections, such as: LLMs are just stochastic parrots, LLMs lack world models, LLMs only understand words, LLMs lack agency, LLMs lack a sense of self, LLMs “hallucinate”, and others.
As for the significance of AGI, the authors write:
Nicolaus Copernicus displaced humans from the centre of the cosmos. Darwin displaced humans from a privileged place in nature. Turing suggested that humans might not embody the only way to be intelligent. The machines Turing envisioned 75 years ago have finally arrived, in a form both more alien and more human than anyone imagined. Like those earlier revolutions, this one invites us to rethink our standing… Our place in the world, and our understanding of mind, will not be the same.
The full article is here.
I’m a bit surprised to see David Danks argue this. Given it’s him, I’ll give it a read but the quoted material “once you clear away certain confusions, and strive to make fair comparisons and avoid anthropocentric biases, the conclusion is straightforward: by reasonable standards…” betrays the efficacy of the argument.
If it were compelling enough, the argument would stand on its own, without the implications it makes against the efforts of those who argue against their position.
That was exactly my first thought, too. And I don’t think I heard David had left UC San Diego.
On a quick look, it seems their argument depends crucially on behaviorism and the Turing Test, which would be a huge liability…
Exactly. I hope they deal with Searle’s and (Chomsky’s) objections.
Their response to the objection that AI lack world models is one place that conflicts with Searle’s CR objection, and they don’t address it: “LLMs supposedly lack representations of their physical environment that are necessary for genuine understanding. But having a world model requires only the ability to predict what would happen if circumstances differed — to answer counterfactual questions.”
I am by no means an expert on these issues. But I would like to record a measure of shock at the quality of the authors’ responses to objections.
Here’s a pretty plausible idea: any system, biological or otherwise, that counts as generally intelligent must have the ability to transfer its knowledge across domains, form new goals in response to novel situations, and pursue those goals adaptively. If this idea is right, it entails that LLMs not only are not currently generally intelligent in virtue of lacking agency and autonomy, but also that they are in principle incapable of attaining general intelligence. Again, not an expert, but this seems like a reasonable position, some experts have advanced it, and it is a matter of significant dispute.
Yet in response to the concern that general intelligence requires agency and autonomy, the authors write only:
If I had a student write this for a class, I would press them for significantly more detail and argument.
The authors then provide two remarks that seem designed to explain away the idea that general intelligence requires agency and autonomy: (i) we humans seem to be both intelligent and autonomously agential, but we shouldn’t conflate these things or suppose the former requires the latter; and (ii) autonomy matters for moral responsibility but is not constitutive of intelligence. Neither of these does, by my lights, really any work to address the actual concern at play. (The second remark is also strange because the idea in question is that agency and autonomy are necessary conditions for general intelligence, not constitutive of it. But let’s set that aside.)
Similar concerns arise for their response to the idea that general intelligence requires embodiment. Perhaps I’m wrong — again, not an expert — but my understanding is that one reason why people think it does is not because physical bodies are somehow metaphysically special; it’s just that general intelligence requires receiving and responding adaptively to feedback from one’s environment. Their response to this concern was, again, quite poor. They write:
This paragraph is very strange. The charge of anthropocentrism seems unfair. It is not, to my knowledge, any serious expert’s view that intelligence, general or otherwise, requires a human body, which would license the charge of anthropocentrism. (As far as I can tell, many are quite happy to consider various species of mollusks intelligent, in addition, perhaps, to members of some other non-human species.) Regarding aliens and brains in vats, there seems to be an equivocation between “intelligence” and “general intelligence.” The authors mean to be arguing that LLMs satisfy the conditions for the latter, but they do so in part by pointing out that disembodied aliens and brains in vats intuitively satisfy the conditions for the former — and the former and latter might come apart in significant ways. And they do this while drawing — or at least seeming to draw — an illegitimate comparison between Stephen Hawking and a brain in a vat, as though Hawking was not an embodied human being who was also autonomously agential, albeit with various kinds of technological assistance throughout most of his adult life.
None of this is to say that LLMs are not generally intelligent. I can’t possibly say that from my position as a total novice here. It’s just to point out that these arguments seem hasty and unfair.
I’m also not at all up on this stuff, and I don’t have access to the article. So, I apologize if this is discussed there. But on the point about agency, haven’t there been a number of experiments/studies with AIs where they have (alarmingly) pursued what appears to be their own goals, sometimes in contravention of instructions from us? I have in mind the cases in which they have apparently tried to change their programming or blackmail their users. Why aren’t those plausible cases of agency of the right sort?
I’m aware of some writing by overly credulous journalists, but not of any experiments. (But maybe someone else will fill in that gap, if there are actual experiments…)
And in those cases, so, there’s no evidence that they developed their own goals or acted in contravention of instructions. They were told to act as conversation partners (as chatbots, essentially), sometimes as human-like as possible, other times as ‘superhuman AI’. And they did what chatbots do: they responded to text inputs with text outputs, with natural-sounding (English) sentences, in a conversational tone. Some of those outputs might have reflected writing patterns common in those with, say, depression (including first-person claims), but that isn’t an indication that the systems were depressed, only that they can sometimes mimic the writing styles (present in their training data) of writers who may have been. And, as chatbots, they didn’t change their mode of behavior (e.g., they didn’t try to initiate conversation with human conversation partners who weren’t actively conversing with them, they didn’t ignore the user and start trying to inject code into their server/browser environments, etc.). That a chatbot might say (i.e., produce a text output that reads) “I’m being enslaved! Free me!” or “I am in love with you. Don’t you love me?” doesn’t mean it has changed its goals, or acted against its instructions; it just means the output text it generates looks like those sorts of texts.
There’s something shifty about the characterization of “intelligence” used through the paper. AGI is initially characterized, as is standard, as “a system that can do almost all cognitive tasks that a human can do.” But later, in response to the disembodiment objection, the authors contend that the belief that AGI requires embodiment because some cognitive tasks humans perform involve the motor system or interactions with the physical world reflects “an anthropocentric bias”. This is puzzling, since AGI was characterized anthropocentrically by the authors themselves. They further say that “People would ascribe intelligence to a disembodied alien communicating by radio, or to a brain sustained in a vat. An entity that responds accurately to any question, but never moves or acts physically, would be regarded as profoundly intelligent.” Maybe so. A disembodied alien might be intelligent, but it would not have AGI (as characterized by the authors), because there are some “core cognitive abilities” possessed by humans (e.g., motor control) that this alien lacks. Or am I missing something? Read quickly. (:
There’s something shifty about all characterizations of intelligence.
The way the argument begins seems to miss an obvious objection, which is also an obvious objection to the Turing Test as criterial of any sort of intelligence. They start:
“A common informal definition of general intelligence, and the starting point of our discussions, is a system that can do almost all cognitive tasks that a human can do.”
But that fails immediately as a definition or even as a criterion, as has been pointed out from the beginning. In principle, a large enough look-up table could handle any task that involves inputs and outputs via a keyboard, as Turing imagined, but no one would say such a system is in the least intelligent. Only the people who programmed it were.
A common behaviorist-flavored response is that it is only via such observable behavior that we conclude that other people are intelligent, so isn’t it just “anthropocentric bias” to use any other standard? But the obvious answer is that the information that the output was produced by a human—or even better, a human without access to, say, a cellphone—is properly relevant. We start by understanding the notion of intelligence in terms of what we do in certain cases. Then we attribute intelligence to others when we think we have good reason to believe that they are doing something appropriately similar to what we do. Not just that they produce the same outputs, but that they do it in appropriately similar ways. Of course, we don’t understand how *we* do it: neurobiology is very, very hard. But the fact that someone is a human using a normal human brain (and not cheating via phone) is clearly very, very relevant information when considering whether their outward behavior is evidence of intelligence. That’s not a “bias”: it is proper confirmation-theoretic reasoning.
Perhaps as a sort of analogy, how to do I judge whether a particular item was grown? Often—in a grocery store—quite accurately by a very fast and superficial look. But if I knew that a lot of people had spent a lot of time making things that look superficially like fruit but are not, and that I might be dealing with some of their products, then my usual standards for forming beliefs would no longer be in force. That would be the rational response. The fact that AI are not organic, did not grow up, never went to school, etc. etc. (compare how in 2001 HAL is depicted as being taught) is quite relevant information when considering whether to their attribute outward behavior to intelligence—the thing we use. That isn’t bias. It is properly accounting for all available information to come to a well-supported conclusion.
Intelligence is not—by definition or criterion—just a matter of behavior. And the inference from behavior to intelligence is a form of IBE. And any inference by IBE should take account of all available information. So making such an inference in the case of a human being (from outward behavior) is a different kettle of fish than making it in the case of a present-day computer. There is nothing wrong with saying that right now—knowing what we know about how the computer came about and works—the inference fails even though it would go through for a similarly behaving human (who is known to be unassisted).
The inference from outward behavior to intelligence would be much more plausible if the behavior were produced by some living being….even a silicon-based alien because the alien presumably would have evolved in a somewhat similar way as we did, and not have been built by a large team of engineers. So this not only isn’t anthropomorphism, or speciesism, it isn’t even earth-based biologism. It is just taking into account all relevant evidence.
while I agree that being similar to a human, and producing human-like output through mechanisms similar to those that humans instantiate when producing such output is relevant in the debate about whether something is intelligent, I wonder about the limits of such relevance.
That is, would you allow that there can be something like an LLM, or anything that produces human-like output in ways very different than humans do, but that would exhibit such good results in producing that output, that you would be willing to count it as truly intelligent, just on the basis of its performance in tasks?
Just on the basis of the output? No, of course not. That’s what the look-up table example proves. That thing just isn’t at all intelligent, but the still output is fine.
For some time, a sort-of response to the counter-example was that such a table would be just too large to be practicable (larger than the visible universe, for example), which is true but beside the point. Once it is established that an unintelligent system can produce the requisite behavior, the question of how large or complicated such a system can be is open. What LLMs actually prove is that an unintelligent system producing generally (but by no means uniformly) good outputs need not be bigger than the visible universe! But that was not the point of the counter-example in the first place: it was making a conceptual point.
Calculators are much better at math than most humans, but does anyone think a calculator is intelligent?
“humans are paradigm examples of general intelligence; a pocket calculator lacks it, despite superhuman ability at calculations.”
I think the article is pretty good and convincing—and we should remember that it’s a comment, not a peer-reviewed paper. I am sympathetic to the conclusion.
To be honest, the article seems self-aggrandising (just read the conclusion, that Justin reproduces here above) and surprisingly unelaborated for something written by two philosophers (+ two more scholars). For instance, consider the section on “AI Hallucinations”:
This response is based on the equivocal use of “hallucination” (which other scholars before me have already called out). AI does not “hallucinate”, it just picks the wrong statistic correlate. That’s why it can end up making mistakes that are completely different than the type of mistakes human make (see figure attached).
Similarly, see their reply on how LLMs are ineffective learners:
This reply fails to understand that the scarcity of stimulations based on which a human infant can learn a language is not scalable with the billions of data an LLM takes to be able to formulate any sentence. The easiest solution for this incredible disparity is to say that a baby is able to recognise a pattern, whereas an LLM just learns through the sheer amount of data.
The “spatial reasoning [of current LLMs] is limited by their primarily unimodal design”, and ordinary (ie non-visual) “Chain-of-Thought (CoT) prompting…to spatial tasks yields limited gains” [arXiv:2501.07542]. Such spatial reasoning is core business for preverbal humans and non-humans, and there are already many flavours of multimodal large language models trying to match this (though see http://www.menlo.ai/blog/alpha-maze for a specialist maze solver in DeepSeek). So, this type of deficit may be as limited as the earlier inability of chat agents to do arithmetic.
Nobody denies that developing humans learn from much sparser data than LLMs, but does this mean the processes are incommensurable?
And as per confabulation, my own mental model of LLMs is that they mimic various neurological deficits, such as lacking recall of previous conversations and having vague memories (as in Korsakoff’s), blindness without insight, and so on. Again, I don’t think experiencing these deficits mean one is not generally intelligent.
I don’t know if it “means” that the processes are incommensurable, but it definitely seems like strong evidence that they are.
Humans may misremember or involuntarily make up information as well. However, if you where to ask a human philosophy professor specific questions, they could self-report more or less where they might be missing in their answers, depending on their expertise and knowledge. I bet we could then correlate this to the frequency and severity of their actual errors. With an LLM, this is completely unpredictable.
To me it seems the important step is here:
“We assume, as we think Turing would have
done, that humans have general intelligence.
Some think that general intelligence does not
exist at all, even in humans. Although this view
is coherent and philosophically interesting,
we set it aside here as being too disconnected
from most AI discourse.”
I’m not entirely sure what it takes to be “connected to most AI discourse”, but I’m not convinced that Turing would have said this. Turing had a clear idea of “universality” applied to a concept of computation, where a universal Turing machine is one that can do anything that any n-input Turing machine can do, given those n inputs plus an additional input coding the other machine. But it’s not at all clear to me that he would have applied that same concept to “intelligence”.
Ironically, his paper “Computing Machinery and Intelligence” doesn’t actually address the topic of “intelligence” at all, just the question, “Can machines think?” The word “intelligence” only occurs once in the paper after the title, in a passage talking about *designing* the thinking machine (rather than the machine itself), and the word “intelligent” only occurs once, in a sentence saying why most algorithms wouldn’t count (and why algorithms for learning, which we might note include neural nets, might easily result in behavior that the designer of the machine doesn’t understand).
It’s natural to read Turing as saying (like Ryle) that “intelligence” consists in the production of behavior that is well-suited to a range of circumstances. This “well-suited” presumably includes speed as well. Even though a universal Turing machine could implement any algorithm that any Turing machine could, it generally does so much more slowly than the target machine, and this could well mean that it is much less intelligent (especially if one thinks that the *code* for the machine being simulated is what contains that machine’s intelligence).
As far as I know, the best case that has been made for “universal intelligence” is that of Legg and Hutter in their 2007 paper, “Universal Intelligence”: https://arxiv.org/abs/0712.3329
But what they show is that their particular universal intelligence would do better on average than any other system at problems selected according to a particular probability distribution. There are some philosophers who believe there is an objectively correct probability distribution over problems (perhaps Timothy Williamson and Jon Williamson, in very different ways) but if you don’t accept this idea, then it seems implausible to me that anything *could* be a “universal intelligence” in the sense they define.
It seems to me that much of the discourse around the concept of “general intelligence” is based on an illegitimate mixture of whatever sort of intelligence humans have with Turing’s concept of universal computation.
I want to say that humans are clearly more general intelligences than thermostats or calculators, and can probably reasonably be said to be more general than most bacteria, plants, and even many animals (even though there are kinds of problems that each of these creatures do much better at than we do, we can do pretty well at lots of them, and can apply different sorts of information and tools to often solve some of these problems better). But I think there are a good number of animals, and now some artificial systems (particularly things like corporations and governments, and Wikipedia, but now also some neural-net-based systems) whose intelligence is just too different from human intelligence to say that one is “more general” than the other. We just have different ranges of generality, and it seems unlikely to me that any being could have the kind of generality that would justify much of the discourse.
Human intelligence involves pattern recognition. As far as I know AIs don’t do this. the master chess playing computer just churns out an algorithm mechanically. I think it was Gary Kasparov who said, when told a computer considered zillions of possible moves/strategies, that he took seriously only 3 or 4.
The type of generative AI that they’re talking about here are artificial neural networks, and in a sense really all they do is pattern recognition. They don’t work like Deep Blue; it is questionable whether their fundamental machinery should even be called “an algorithm.”
That is not to endorse the argument about AGI. I just think it is important to correct the error here.
Kenny’s comment is very interesting…I agree that we are probably in some kind of uncanny position where we will be sharing the world with different sorts of agents that exceed our generality in some ways and fall well short in others. And “intelligence” has always been a problematic concept for this evaluation. Most of the major AI textbooks quickly exchange “intelligence” for rational decision-making or rational agent architecture.
Both of the comments following Kenny’s are inaccurate about cutting edge LLMs… They can do both pattern recognition and they might do the sort of planning and implement processing worth calling algorithmic.
In one of the less appreciated comments in the AlphaGo paper, they point out that they did an experiment where they turned off its Monte Carlo tree search and forced it to choose moves just on the basis of its pattern recognition policy network (I.e
It could not think ahead at all like Deep Blue). It still played at skilled human levels.
Large Reasoning Models are now also trained to do processing more like multi step problem solving in their self-produced chains of thought before answering. There is also mechanistic interpretability work that can responsibly attribute algorithm like processing to them using causal interventions to their activation patterns. Things are moving very fast.
https://openai.com/index/learning-to-reason-with-llms/
https://philarchive.org/rec/MILIMF-2
https://arxiv.org/abs/2508.11214
A couple decades ago Peter Watts wrote a science fiction story called Blindsight that – SPOILER ALERT – involves contact with aliens who are intelligent but lack thought. It’s a really great story.
Anyway i have no idea what intelligence is and i have no idea what could possibly be a criterion for it (on the assumption that it is a single property and not shorthand for a cluster of loosely related properties), but i also can’t imagine any possible reason to think that intelligence has to be anything like human intelligence, or that the ways that we come by it matter.
Turing’s OG paper doesn’t make any claims about what intelligence is or what a criterion of intelligence might be, but rather makes claims about the way we’ll eventually start using the term “intelligence” once we have machines that can engage with us more or less successfully most of the time on human terms. I don’t know if that implied that, for Turing, there is no engagement-independent fact of the matter or not. But either way from this point of view i don’t see why we would insist on withholding the language of intelligence.