Machines Learning to Learn
Harry Baya, June, 2013
This discussion is intended to focus on the possibility of developing a machine based system capable of the kind of learning that would be needed for it to learn about how it learned and then be able to improve the process. This concept happens to be part of Ray Kurzweils view of the possible “Singularity” in the next 20 to 40 years.[1] However, I am trying to focus on the general problem itself, regardless of how long it may take to solve.
My best guess is that my thoughts below are vaguely associated with two main “learning” sequences:
o How this process occurred : no life -> life – > consciousness
o And my guess is that there is a very significant software/social/species-interaction component of this, not just a biological/hardware evolution of abilities.
o How infants learn to think, act and talk during the first few years of life.
One line of thought:
Embodiment is important. The entity must be mobile and have something like two hands. My first guess is that it would need
o a head with eyes and ears
o a neck (with the ability to rotate and to tilt to permit viewing in any direction.
o a body (central mass, center of being.. possible mass)
o two “arms”. each arm would be like the neck – possibly two bones (would one do), ability rotate and angle in any direction from the joint
o two “hands” : The hands could consist, initially of two opposing “fingers” that could grip things.
o Initially I might use wheels for mobility and avoid the complexity of joints, feet, toes etc.
o The surface of the body would be touch sensitive everywhere… perhaps greatly heightened in the hands.
Initially it would understand how to do a very small set of things, along the lines of :
o go (move in x-y coordinates) – possibly just forward.. no dimension
o stop moving
o move a limb to any angle and rotation from a joint (note getting a hand to a particular position and orientation could be quite tricky… again.. maybe the initial arm would be a simgle “bone”…. though this does so limit the ability to reach around things etc.
o Close/open fingers with an intended number of degrees and a resistance setting so as not to crush things being grabbed. (marshmellows vs blocks of lead )
The idea is to use the above and a program to have it “learn” to do things
like :
o go to a location ,
o touch something
o find something by touch.
o pick something up
o put something down
More on embodiment: It seems to me that much of our language is based on metaphors related to physical space and movement. “Getting somewhere”, “Making progress”, “falling behind”, “getting on top of” etc. There are thousands of these word sequences that could be related to physical space and motion but are used to discuss abstract ideas. My guess is that these expressions and related meanings would be most likely to evolve into the mental structure of a machine if that machine had a body and was able to move and do things in 3-d space. It is possible that the machine could a virtual machine in a vitural space… but this does raise a fairly formidable programming challenge, i.e. Can we create a virtual 3-d world rich enough in experience for virtual machine to learn to “think” in a way that would be useful in the real 3-d world.
Here is another way of viewing the same problem and tradeoffs:
Though the obvious environment for this would be a real-world room with walls and objects in it, it might be possible to greatly speed up the learning if it could be a simulated world with a simulated robot with all of the attributes described.
Creating the base set, real or simulated, would be a fairly good size challenge. However, I see nothing in what I have described that is not well within the ability of current technology.
The challenge… the real work related to the challenge/goal discussed in this essay.. is to create a program (or whatever this virtual control/thinking system is best labeled) that would permit learning to occur along with following instructions from outside. Though I say “a program”, I actually think we should be seeking as many different programs as possible. Competition for survival (meme like intelligent consciousness) could be an important part of the process. However…the systems themselves would eventually have to decide how many competing systems would be allowed to exist and how the resources would be allocated between them. “Evolution” in the broadest sense did this for life and intelligence on Earth.
The idea is that the program would remember everything in its experience and would form concepts based on actions and rewards and punishments. We would teach it to do things and it would remember how to do them.. but, our real aim would be to teach it to “learn” how to do things.
Note that this vision involves a human teacher (or many) working with the robot/mind to help it “learn to learn”.
My guess is that this is somewhat parallel to what goes on in babies after birth.
xxxxxxxxxxxxxxxxxxx
I would love to turn this into a funded project of a student in an AI lab and have someone pursue it. I suspect similar things have been done and the first step would be to research them and find out what they did and what success and failures they had.
I would like to refer to the Scientific American article that started my thinking, many years ago, along this line. [ 2 ]
XXXXXXXXXXXXXXXXXXXXX
The system/robot would record every action and every bit of data received. Actions sequences that are repeated (pattern recognition.. search for similar sequences) could be given identifiers ( higher level concepts) and recalled, repeated, or used as a starting template for a new, slightly different action.
This is one of the key concepts in all this. The context, either real or virtual, is necessary and is critically important. However, I don’t think that will be a challenge. The challenge will be to create even a very primitive learning process that can somehow create/identify/recognize patterns of actions that can be recalled and used in “similar” situations in the future.
This is wide open space here. Though there will almost certainly have to be a few pre-defined “concepts” to start with, the idea is that these will be a very limited set of fundamental building blocks needed to permit the development of other “concepts”.
My guess is that a lot of relevant work has been done on pattern recognition that would be of value in this approach. The idea of “generalizing” in the sense that two patterns are similar in some way and can be seen as instance of higher level pattern/template is important to the approach I envision.
For example turning left is a different action from turning right, but we would like the system to recognize that both are examples of turning. The eyes mounted on the front of the head will see what is in front of them. As the head is “turned” the view will change. Could the system recognize the similarity between turning the neck/head and turning the body? The neck would permit the head to look up, turning in another dimension… more generalization.
In some ways, to be honest, I have no idea what I am talking about here. I don’t have names for the “learning functions” I want the machine to “grow” (evolve, build, recognize /? ). I don’t even have a strong enough grasp of the perspective to say that it is even a viable approach to this general area. It’s too abstract. It describes a behavioral goal while giving little or nothing that suggests how it might be attained. I see that..yet, I cannot resist trying to find something of significance while reflecting on the problem.
Yet I feel that I have a least a little insight into something that could be significant. If this is so, the odds are that others far more qualified than I am have already explored it. That’s OK. I don’t mind re-inventing the wheel and, more importantly, so far as I know their work did not result in an open ended learning machine like I envision. So it’s not only “OK” to cover ground that is already covered… it may be necessary because they may have (apparently did if they are at an ending with no success) missed something along the way. Each researcher exploring the conceptual space with a “beginner’s mind” offers the possibility of finding a solution.. or a contribution to the process that will someday lead to a solution.
The key parts of my concept are:
o total memory of all actions and all input data
o The ability to find repeated, or similar patterns in this data. The “generalization” engine. How does the system decide what to generalize? Can the system learn to generalize about the process of generalization?
o Some sort of reward/punishment process, initially run by a human teacher, that seeks to induce the system to start building “concepts” (patterns, perhaps generalize as templates that would include a large family of pattern) from the stored memory of all actions and all input data. Part of the input data is the positive and negative feedback from the human trainer.
One of the models for this in my mind is the process that a newborn infant goes through in learning to use his or her hands to pick things up, to recognize objects visually and distinguish one object from another. Infants eventually “learn” that there are such things as words.. said by older humans that “mean” something. Here we get to what “mean” means. At a simplistic level “means” would mean [ sorry , best word ] that a certain word or pattern of words would alert the hearer that some outside data input, such as an action, such as being fed, was likely to happen
The development of useful “concepts’ (identifiable patterns or classes of patterns) would be part of a built in preference to increase the number of actions that would increase the likelihood of a reward, or decrease the likelihood of a punishment.
To say that this is a difficult challenge is an extreme understatement. It is not only difficult it may be impossible in the foreseeable future. It may be that some of the software (which implies the concepts and understanding on which it is based) needed to create a primitive learning system that could become increasingly sophisticated (complex, higher level concept/generalizations, pattern recogntions including past actions and all past input data), for ever will not come into being for many years.. perhaps thousands. Yet I still feel that this approach is viable in some sense and implies to me (the clueless dreamer) that a true “learning machine” can be created such that at some point it will be able to continue learning without a dedicated teacher. This is the case with humans now. It seems likely to me that a similar machine based learning entity can be eventually be created. Once it really”starts”, it will be able “live” in our world and continue to learn. Though each one of these “beings” would develop it’s own set of :”concepts”, they would be able to communicate with each other and may be able to find commonalities that would be of great help to any beginning “learner”.
As I see it this is how the human consciousness came into being over millions of years. Concepts were formed and eventually the humans began to communicate more and more information between themselves. The word vocabulary grew and knowledge could passed more efficiently to another human, and to the next generation. Along with the word vocabulary the conceptual framework, the set of “things” a mind could think about, was also growing. Concepts like the one listed below probably became part of human thought before there were words for them…or maybe they came into human awareness at the same time, Maybe the same process that created a word for something created the generalized concept which the word labeled.
up, down, more, less, fear, pleasure, fall, throw, get, fight, die, teach, learn, discover
These are the kinds of things I want my “learner” (machine based intelligent system) to be able to discover/identify on its own The initial building blocks and the teacher are just a starter needed to get the process started. They should be as sparse as possible, the fewer the better.
It may be that the critical design aspects of this process can be discovered by working with a much simpler, simulated, environment. Perhaps something as simple as the computer simulation/game called “life”. [http://en.wikipedia.org/wiki/Conway’s_Game_of_Life ]
Once the code that will support true “learning” is developed it should be able to learn how to make itself better at learning. This is part of what Kurzweil thinks will happen at the singularity in 20 to 40 years. What triggered my thoughts that resulted in this essay was the statement that Kurzweil’s hope/expectation was never going to happen because no real progress had been made in developing even a very simple learning system that could seem to have the possibility of learning enough (which would include the changes it could make to itself) to improve itself forever. I argue, and attempt to show in this essay, that this problem can be solved even though it is a real dilly.
It’s the “never” part that offends me. If they are really just saying that it may take a very long time, I agree. It may. However, it seemed to me that the critics were not just saying that it won’t happen anytime soon. They seemed to be saying it simply was not going to happen. I disagree with that statement.
The evolution of today’s mathematics (e.g. the cutting edge frontier of theoretical Mathematics as well as the most advanced approaches available in applied mathematics) from pre-verbal humans seems almost impossible, but it did happen. Creating a true “learning system” as I have tried to define it above, may mean overcoming a gap as broad as that evolution (pre-verbal - mathematics) and could take as long, or longer. But, I don’t see why it cannot be done. Humans did it, why can’t computers? [ for a very interesting book describing the growth of mathematics from early humans to the 20th century see “Number: The language of Science” by Tobias Dantzig (1930).]
I would also like to comment on a similarity I see between (a) my idea to create a learning process with as few “givens” as possible and (b) early machine languages. I want to have a few a number of pre-set “concepts” as possible. What do newborns have?
I note the similarity between this need and the sparseness of the earliest machine languages used to program the CPU of early computers. I recall learning an assembly language (I think it was for the IBM 1130 or 1620) that had less than 10 primary commands. It had two or three memory registers and the ability transfer a bit pattern from a CPU register to an addressed location in a big memory (RAM) . It could also get a bit pattern stored in a memory address and put it into a CPU register. The commands I recall were something along the lines of:
o store in memory from register
o store in register from memory
o clear register
o transfer the contents of one register to another register
o add the contents of two registers and put the answer in a third ( I think binary “and” and “or” was part of this and the subtract.)
o subtract the contents of one register from another and put the result in a third
o compare the contents of two registers and put the result in a third ( there were three possible results : bigger, smaller, and identical)
This very limited set of commands permitted building sequences of steps (commands, executable statements) for things like multiply, and divide. In fact this limited set of commands permitted building languages like Fortran, Basic and “C”. The very high level concepts that were part of this included:
o stored data
o stored executeable commands
o the ability to get and execute a command from a stored location, along with a way to determine where to get the next command. ( bringing with it the possibility of an “if” statement)
o Loops – repeating sequences of statements
o If statements (or other conditional statements) that would change the sequence of statements based on whether or not a particular “condition” (such as register value exceeing the value in another register)
o Stored “named” sets of statements (subroutines, functions, sub-programs,modules)
My best guess is that fundamental learning of the type I want machines to start doing, and build on, will begin with a similarly sparse set of operations.
Could Pythagorus have envisioned modern computers using the above commands to create complex programming languages? I don’t know? How about the best mathematicians of 1000 years before Pythagorus, 10,000 years earlier…etc At what point was this vision possible?
From that perspective, where are we with creating learning machines? Are we still thousands of years from having the conceptual framework needed to do for learning what the creators of digital computers did for computing machines?
The concept of a “bit” as a useful way of storing information is recent. There was not need for anyone to think about it. I read a book [ 3 pages 14 & 15 ] on mathematics in which the author discussed number systems to different bases (e.g. the decimal system is to the base 10). He mentioned in passing, that it was even possible to work with a binary number system, to the base two and that it would have only two digits, 0 and 1. I inferred that the author felt the binary system was far too cumbersome to use and was therefore of no real interest. He did mention that one famous mathematician, Leibnitz, was quite taken with binary arithmatic and saw in it the “image of creation”.
The mathematician, Tobia Dantzig, was on the other side of the jump into wide use of binary arithematic as the foundation of most computing systems. Similarly , we are on one side of a jump to where machines can truly learn. We may have the necessary tools available now but not recognize their usefulness, as was the case with binary arithamatic.
We, like Dantzig , are on one side of a huge jump. In our case we would jump from dumb machines to ones that can truly learn, and learn in such a way that they can learn to make themselves more efficient at learning.
The design/architecture concepts we need to accomplish this task may be a visible to us as the binary number system was to that mathematician, and may seem useless to us, just as a binary number system did to that mathematician.
I am suggesting that we may already have the tools we need to create a learning machine of the type envisioned above. I think it would interesting to assume that we do indeed have the tools, and that we just have to figure out how to use them. I don’t suggest “faith” that we can do this. I don’t see that “hope” would interfere with the process. I suggest that it’s worth pursuing. Even if we are millennia from real success, that doesn’t mean that we can’t learn some very interesting things while trying to solve this puzzle.
I repeat: I don’t have faith, or belief, that this problem can be solved in the near future (next 100 years), or in any reasonable amount of time (less than 100,000 years). Rather, I think there is the possibility that it can be solved and I think pursuing this goal would be a rewarding adventure from now until it is reached.. however long that may take.
I also assume that there are numerous instances of intelligent life spread throughout our universe. It seems likely to me that if what I call “real machine learning” (where machines can learn to manage their own evolution in such a way that they at least reach our level of consciousness, and possibly (a) accelerate the process and (b) move on to higher stages of awareness) will have occurred many times in other civilizations on other worlds.
I suppose I do have “faith”, in the sense that I think it is highly probable, that the problem can be solved. If nature can support the process of going from innate/dead matter to human consciousness, then why can’t the universe support the same destiny for machines.
And of course the other interesting question this brings up, both for humans and the learning machines I forsee, is.. and then what?
[ lifeless matter ] ->
[life ] ->
[Human Conciousness & possible machine consciousness] – >
[ and then what ] ->
[ and then what ] ->
…
[1] Ray Kurzweil’s article on “The Singularity” in Wired magazine:
http://www.kurzweilai.net/wired-live-forever-and-save-the-world-ray-kurzweils-new-singularity-university-will-change-how-you-think
[2] My earliest exposure to this general area was a book a friend’s father gave me while I was in high-school. The book was called “Automatic Control” and was published by Scientific American in 1955. I still have a copy. It collected together articles related to what we now call computer science. The one article that dealt most directly with the possibility of machines that could learn to improve themselves was “Man Viewed as a Machine” by John G. Kemeny. John Kemeny was then a 28 year old professor at Dartmouth college. He went on to become an important contributor to computer science. Among other things he was one of the lead developers of the BASIC language. In the article he discussed machnines that use a set of plans to build a machine. He then pointed ot that the machine could be given plans for itself and then be able to reproduce. If random changes were introduced into the design we can foresee the possibility of evolution.
I recall reading another article in which this concept was used in an imaginary environment in which reproducing machines were left in a closed room/building with a limited amount of material from which to construct new machines. They would then mutate and create and eventually would have to compete with each other for material to build offspring. If they could “conquer” another machine, they would use it’s parts to create offspring, with mutations. The article projected that given long enough, these machine would evolve something like consciousness and intelligence because it would give them a competitive advantage.
[3] “Number: The language of Science” by Tobias Dantzig (1930).] This is a fascinating book that is intended to assist someone with little or no mathematical training to understand the most important concepts in Mathematics up until the time of the author.