Monthly Archives: July 2020

The definition of intelligence

When reading AI papers I keep running into definitions of intelligence. Two researchers – Shane Legg and Marcus Hutter – even made a nice effort and put together a collection of them [1]. I don’t know how about you, but I keep finding them unsatisfactory. Apparently, a popular and widely accepted one nowadays is

“Intelligence measures an agent’s ability to achieve goals in a wide range of environments”

by Legg and Hutter (L&H) [1,2]. It sounds ok and yet – do you feel any closer to understanding what intelligence is after seeing it?

Intelligence definitions suffer from various common maladies. Putting aside that many people just don’t understand what intelligence is, there are two main reasons for their inaccuracy. One reason is a bias towards circumstances. The authors are not trying to be accurate, but instead they are tailoring their definition to their specific needs. Others (perhaps unknowingly) conform to whatever the opinion of the public or scientific community, or research direction, is. In other words, there is a divide between what intelligence is, and what people expect it to be.

The other issue is a prevalent logical inaccuracy. Generally, a proper definition needs to have two main properties: to completely cover what we want to describe, but also exclude everything else (the third property being that it is simple). But with the existing definitions that is, to my knowledge, never the case.

Many describe intelligence too explicitly, in too much detail and using examples. That is especially common for the older ones and ones done by psychologists (who are rather practical and human oriented than formally accurate). One example, picked at random from the collection:

“the general mental ability involved in calculating, reasoning, perceiving relationships and analogies, learning quickly, storing and retrieving information, using language fluently, classifying, generalizing, and adjusting to new situations.” Columbia Encyclopedia, sixth edition, 2006

The result is a definition that is perhaps good for uninitiated readers, but is too constricted to describe all that we want to understand as intelligence. For our needs it does not suffice to describe human intelligence – we are dealing with prospects of future AI’s and perhaps also extra-terrestrial ones. So we need to define it even more broadly than we need right now.

On the other side, many intelligence definitions suffer from being too loose and including too much. A good example is a definition by Minsky:

“Intelligence is the ability to solve hard problems”.

It indeed is. But there are other things too that can solve hard problems. Like a pneumatic hammer. Or a brute force state search. Are those intelligent? No and not much.

What are we trying to define?

There is a lot of confusion about what intelligence is, and what level of it is enough to call something intelligent. This stems from the fact that different people have different subjective experience, expectations and applications for it, and nobody has properly defined the intelligence itself yet. What matters for most people is human intelligence and how to compare it between people. Some are trying to find where on the scale animals end and humans are. Others are working with AI, which works quite differently, while, on the applied side of the research, is still being compared on the same scale and the limit of what already is intelligent and what is not is attempted to be specified – without much success due to insufficient understanding. There is this funny property – “When it starts to work, we don’t call it AI anymore” (this is often quoted but I can’t find an attribution). The theoretical scientists and philosophers are attempting to find a clear and generic definition free of all the clutter.

The point here is that there are very different expectations and applications to match – both theoretical and practical. Different people want different aspects of intelligence to be emphasized and detailed while others can (or should) be kept simple or omitted. Therefore it would be a mistake to try to fit one definition on them all and attempting to do that is one of the reasons why past researchers have failed.

What I propose is, instead of writing one definition, creating a framework with a simple core that can be extended for the specific needs.

Before presenting it, I will first show how the definitions are constructed (and pinpoint some errors) which will lay foundations to the new framework.

Modularity

Nowadays enough research has been done and enough terms defined that making a proper definition is not an artistic endeavor anymore but rather a mechanical work of grabbing available pieces and plugging them into a frame to achieve the desired outcome. I will demonstrate this on decomposition of the contemporary definition so that it is more clear later.

“Intelligence measures an agent’s ability to achieve goals in a wide range of environments”.

1) “Intelligence” – the subject, necessary.

2) “measures” –  “is” is commonly used too. “Measures” stresses that it is a measure, therefore a range and something that can be measured.

3) “ability” – it is a property of something and it enables something.

4) “to achieve goals” – it has a target, as opposed to properties that just exist without any direction at all. Note that this is not sufficient for purposefulness. Evolution has a goal (gene spreading) but it does not reason and has no purpose. I think that having a purpose is not necessary for intelligence though.

5) “agent’s” – intelligence is a property of something that has agency, acts. Not strictly necessary, but without agency  the intelligence would be inconsequential.

6) “in a wide range of environments” – this is the main contribution of the authors and the meat of the definition. The authors believe that this is a sufficient prerequisite for intelligence as it implies a wide (… full) range of intelligent abilities. To quote from [2]:

“Reasoning,  planning,  solving  problems,  abstract  thinking,  learning  from  experience and so on, these are all mental abilities that allow us to successfully achieve goals. If we were missing any one of these capacities, we would clearly be less able to successfully deal with such a wide range of environments. Thus, these capacities are  implicit  in  our  definition  also.”

True. But so does having legs or a lot of money. While the success in a wide range of environments is a good addition to intelligence, it does not define intelligence. It only defines versatility. To me it seems that the reason why this definition came to existence and got popularity is the current research which is trying to shake off the disappointment of AI’s that were supposed to be the end game but instead turned out to be “narrow” and useless for anything but their specific application. Therefore the focus today is on “general” AI, which is exactly what this definition aims at. So while it looks great by being very general and simple, by being too general it violates the second property of a good definition and fails to define intelligence. Which, after all, the authors admit themselves in the end. “We simply do not care whether the agent is efficient, due to some very clever algorithm,or absurdly inefficient, for example by using an unfeasibly gigantic look-up table of precomputed  answers.  The  important  point  for  us  is  that  the  machine  has  an amazing   ability   to   solve   a   huge   range   of   problems   in   a   wide   variety   of environments.”

The definition

What I propose is one core definition of intelligence and then an array of optional extensions to satisfy the specific needs and use cases. The core does not contain anything it does not have to, it is as simple as possible and to the point.

“Intelligence is an ability to process information.”

It intentionally does not say who has the ability, to what end, or to what degree. Because those are already various measures and properties of intelligence that are not necessary to define it. Does this define intelligence? It seems too simple and perhaps counterintuitive. But that is because of the framing we are used to from our perspective in which people are intelligent and chess programs are not. But we need to take more than one step back in order to see the whole picture.

The reason for emphasis on information is that it is exactly what separates “thinking” and “intelligence” from the manipulation of physical objects. Brains are intelligent, hammers are not. Even calculators are intelligent, just to a very trivial degree.

As far as I can say, the definition can’t be made more simple than it is without completely breaking it. So the question rather is whether anything that is necessary for intelligence definition is missing. I have already addressed many such components, such as the agency or goal, but I would like to mention a couple more.

It is tempting to say “ability to process and utilize information”, but even using the information already falls on the “interface” of the intelligence. If you imagine the intelligence as something that is happening inside a box, taking inputs, doing the “processing” and giving outputs, the usage of the information means using the results of the processing and already falls in the space outside the box, or on its border.

The most striking deficiency is that there is absolutely no indication of a measure of the intelligence. I think that it stems from our expectations. We hear about intelligence a lot and almost never think about the intelligence itself, but instead automatically go a step further and are interested in measuring and comparing it. But measuring the magnitude of something is a different topic than its definition. A very important topic certainly! But it is a very complex one that I will not attempt to address – many researchers, including Leg and Hutter, are working on it and making nice progress (by the way, their definition correctly does not address the magnitude either). A related question though is how useful a definition is as a foundation towards being able to measure intelligence. If we could choose between two equally powerful definitions, then the more practical one would be better. But right now the main thing to get at least one definition right – the practical considerations are the next step. I would say mine is as good as any and its design towards modular extensibility is already a step towards practical applications.

As for the optional addons, here are some examples.

  • Agent’s … – if we want to emphasize what our research aims at
  • (an ability to) achieve goals through (the processing…) – to say that we are trying to use the intelligence to solve something
  • Complex (processing) – To emphasize that certain degree of intelligence is necessary in order to call it intelligent
  • … namely calculating, reasoning, perceiving relationships and analogies, learning quickly, storing and retrieving information, using language fluently, classifying, generalizing, and adjusting to new situations. – to tailor it to people
  • … in a wide range of environments – to emphasize we are looking for versatility and to distance from narrow intelligence

As you can imagine, you can create quite anything, including the L&H definition. With the caveat of including the information processing clause – lack of which was my motivation for this paper in the first place. Intelligence is about information, so let’s go from there.

[1] The ultimate definition of intelligence, Shane Legg & Marcus Hutter, 2007, https://arxiv.org/abs/0706.3639

[2] Universal Intelligence: A Definition of Machine Intelligence, Legg & Hutter, 2006, https://zoo.cs.yale.edu/classes/cs671/12f/12f-papers/legg+hutter-universal.pdf

Fair reward – merit or effort?

For us, responsible people, it is clear that just and fair is to be rewarded accordingly to our contribution. If we try harder, work more and better and create more value as a result, we expect to get more in return. And accordingly, if we don’t try or we do a lousy job for whatever reason, we understand that we deserve less for it. An abominable contrast to it is the altruistic system that commands that we shall not ask more for doing a good job. In fact we shall not ask for any rewards at all. The rewards go to those in need, regardless of their contribution or how deserving they are. This system is fundamentally unjust.

While this is clear, many people hold a different view that leads to a very common conflict – and not only among philosophers. A typical objection I keep hearing is the following:

Different people have different opportunities that they cannot affect. Why should that make some people better off than others? Imagine two identical children. One is born into a rich family that provides it with good education and raises it to be confident and successful. The other’s parents are poor and abusive, the child receives little education and grows up to be a nerve-wreck. Both of them start to work and put equal effort into it, doing the best they can. Is it fair that the first receives vastly higher wage and acclaim?
The person telling you this believes that it is fair to reward people according to their effort and not the actual value their work creates.

I have to admit that this does make sense in a way. In line of the ethics we started off with, a person should be rewarded based on what they do. So why should a person be punished or rewarded based on things that are not in their power to change? Rewarding people based on their effort indeed is fair as well. Another perspective comes from the negative side. While I abhor the idea of a poor lazy person getting someone else’s money in welfare, I similarly dislike it when an arrogant moron makes a lot of money just because he was born with a golden spoon in his mouth. Formally, he may create a lot of value, but only a tiny fraction of it can be really credited to him. I find them both undeserving.

How can that be? How can there be two conflicting definitions of a just reward at the same time?
As always – “Whenever you think you are facing a contradiction, check your premises”.
Hint comes when you need to give an answer to the person with their heartbreaking children story. Maybe you have a better one but the best answer I can come up with is – “The world is not fair. It sometimes sucks, but we just have to live the life the best we can with the cards we are dealt.” Which is a lousy answer when trying to explain what fair means.

The reason is that these two cases of justice, while talking about the same thing, are based in different worlds.
Those different starting conditions that our objector complained about are based in the real world. It is the reality we all live in, which is without values or feelings. It deals to everybody, everyday, something different, and there is no fairness in it whatsoever. It is what it is.
On the other hand, the justice we wanted originally applies on a higher, abstract level – on the level of us humans and our ethics. It is the level where values do exist and the one that we can choose and change. So while the conditions each of us is given are unfair, we can create fairness in how we interact.

While this explains how two seemingly colliding definitions of fairness can coexist, it does not say how to deal with the violation of the later. Unfortunately, dealing with unfair conditions is an open problem and previous attempts to solve it have led to some top-tier catastrophes. Formally, the statement of both is quite clear. But the practicality of their solutions differ widely.
The problem is that the amount of “value” we can distribute is limited. We only have as much as we create. Value can’t be drawn out of thin air – regardless politicians often saying otherwise.
Giving a fair reward proportionally to the value created is straightforward. It just means exchanging value for value in a corresponding manner. Value created is distributed back proportionally and without significant issues. So the overall vision that 1) the world is not fair, deal with it; 2) but reward everybody according to their contribution – is simple, clear, consistent, and easy to implement.
On the other hand, trying to fix the unfairness of the world itself and reward people according to their effort is impractical. There is no way to objectively assess an effort a person is making. If somebody creates something of value to you, you don’t need to care how and why they did it – the value, for you, is objective. But knowing how hard they tried? Was it an incapable person doing their best, or a very capable one but slacking, or a one specializing in the skill of acting out a hard effort?
While we can make a personal call and pay extra to a person we know to be good and honest and trying hard even though they did not do so much at the end, this can’t practically be extended to a large scale. Any attempt to do so inevitably fails on the subjective nature of an effort. Moreover, since it can’t be correctly assessed, it only creates wrong incentives for people – to pretend to try, instead of doing actual good work – destroying value for everybody as a result.

“World is not fair” is a poor answer, but currently the best we have. Trying to fix that, on a global scale, should be done with utmost caution as such attempts have already cost hundreds of millions of lives. Until somebody figures something out (naive wishful thinking really does not count), we should stay content with playing the cards we were dealt and the rewards we deserve. Which is not that bad.