ACADEMIC WEBSITE - ADRIAN HORZYK, PhD DSc

AGH University of Science and Technology in Cracow, Poland
Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering
Department of Automatics and Biomedical Engineering, Field of Biocybernetics

• Didactics
• Artificial Intelligence
• Introduction to Artificial Intelligence

INTRODUCTION TO ARTIFICIAL INTELLIGENCE

Preliminaries To cope with the definition of artificial intelligence we have to explain the term intelligence first. Next, we have to think through how intelligence works, and where it is located? What factors is it dependent on? What structures are necessary to enable its development? What circumstances do our intelligence further increased? How the increase in intelligence is dependent on knowledge, surroundings, needs and other circumstances? Where intelligence is located and where it can develop and why? Can we place intelligence in some artificial system or location, where it could work? Can artificial intelligence solve also other tasks than biological systems? Is intelligence limited somehow? Can we somehow widen the computational capacity of artificial intelligence and knowledge to solve much more complex problems than those solved by people?

The definition of intelligence is controversial because many factors of how it works are still not discovered. There are probably as many definitions of intelligence as there are experts who study it, and who were asked to define it [R. J. Sternberg]. The appropriate encompassing definition of intelligence can be given only when we will understand how it exactly works, and we will be able to model it in some artificial intelligence systems. First, the abilites and capacities described below have to be properly understood and modelled in terms of computer science. Intelligence is defined im many various ways:

• Intelligence - the ability to acquire and apply knowledge and skills. [Oxford Dictionaries]
• Intelligence - the ability to learn or understand things or to deal with new or difficult situations; the skilled use of reasons; the ability to apply knowledge to manipulate one’s environment or to think abstractly as measured by objective criteria (as tests). [Merriam-Webster Dictionary]
• Intelligence is defined as general cognitive problem-solving skills. A mental ability involved in reasoning, perceiving relationships and analogies, calculating, learning quickly… etc. Earlier it was believed that there was one underlying general factor at the intelligence base (the g-factor), but later psychologists maintained that it is more complicated and could not be determined by such a simplistic method. Some psychologists have divided intelligence into subcategories. For example Howard Gardner maintained that it is comprised of seven components: musical, bodily-kinesthetic, logical-mathematical, linguistic, spatial, interpersonal, and intrapersonal. Other definitions are: "Intelligence is what you do when you don’t know what to do." "Intelligence is a hypothetical idea which we have defined as being reflected by certain types of behavior." [Brain Metrix]
• Intelligence - capacity for learning, reasoning, understanding, and similar forms of mental activity; aptitude in grasping truths, relationships, facts, meanings, etc. [Dictionary.com]
• Intelligence measures an agent’s ability to achieve goals in a wide range of environments. [S. Legg and M. Hutter]
• Human intelligence - is the intellectual capacity of humans, which is characterized by perception, consciousness, self-awareness, and volition. Through their intelligence humans possess the cognitive abilities to learn, form concepts, understand, and reason, including the capacities to recognize patterns, comprehend ideas, plan, problem solve, and use language to communicate. Intelligence enables humans to experience and think. [Wikipedia]
• and many, many others... [Collection of Definitions of Intelligence]

This general ability consists of a number of specific abilities:

• the adaptability to a new environment or to changes in the current environment,
• the capacity for knowledge and the ability to acquire it,
• the capacity for reason and abstract thought,
• the ability to comprehend relationships,
• the ability to evaluate and judge,
• the capacity for original and productive thought.

Additional specific abilities might be added to the list, but they would all be abilities allowing a person to learn about, learn from, understand, and interact with the environment. Environment in this definition doesn’t mean the environment of the earth, such as the desert, the mountains, etc., although it can mean that kind of environment. It has a wider meaning that includes a person’s immediate surroundings, including the people around him or her. Environment in this case can also be something as small as a family, a workplace, or a classroom. [About.com]

Various types of intelligence at various levels of development can be found and described when observing people and animals.

Intelligence is closely related to knowledge, comprehension, cognition, generalization, creativity, associating, learning, reasoning, thinking, concluding, awareness, and fulfilling of needs. Intelligence is manifested in many various abilities: solving problems, adapting to surroundings (if necessary or profitable) or changing it (if possible), planning, scheduling, developing hypotheses, theorems, rules, principles, strategies, and algorithms. The more developed intelligence is, the more effective it acts, solves tasks, and more optimal results it achieves.

Each problem is hard until it is solved. What does it mean 'to solve problems'? How problems are solved? What processes are necessary to be run to solve problem in general? We will manage these questions also in the next lectures.

There are certainly ways to increase one’s intelligence, also called intelligence amplification or enhancing, by practicing many proven cognitive tools such as mnemonics, problem-solving heuristics, creativity techniques and decision-making tools. An increase in the intelligence level can only result in a better life, health, progress, and standard of living. Below you will find some simple intelligence boosters [Brain Matrix]:

• Deep thinking: in life it’s not enough to just react to events, and situations, rather we should have a conscious objective and select our actions to get nearer our objective. Also it’s important to think about consequences of our actions, to minimize the possibilities of errors and regret. Deep thinking would normally help you live better, and reach your goals.
• Good reasoning: it is the key to success, especially if performed consciously and in the proper order:
  1. have an objective,
  2. make a general sensing about it,
  3. determine your decision based on your sensing,
  4. make alternate plans (along the main objective),
  5. select the best response or a plan,
  6. start by carrying out your plan,
  7. observe results,
  8. store experiences (for future reference).
• Learning from past experience: it is believed that many inventions were actually reinventions; for example Egyptians 2000-4000 years ago were using some unique techniques to build their temples, buildings… but since the early Egyptians were not good at keeping records of what they were doing, many of their inventions and techniques were lost, and they had to be reinvented many centuries after them, which means that we had to start from point zero again because we didn’t keep records of our discoveries in the past. You can apply that to your own life, learning from the past experience either bad or good is very beneficial, and can save you a lot of time and effort. A good way to do that is journal writing which is a useful way to develop self understanding, and to analyze events, in addition to provide a record of how we change over time.
• Practice and experience: you cannot learn swimming from a book, the same thing should be taken into consideration when dealing with “thinking”; you cannot learn to think without practicing. And as mentioned earlier, a good way to start is with cognitive tools such as brain exercises: mnemonics, problem-solving heuristics, creativity techniques, brainstorming, puzzles, brain teasing games etc.

Increase in intelligence is closely related to increase in knowledge and experience in some chosen areas. We behave more intelligently if our gained knowledge increases. This is not true in general but is limited to the areas of things we manage, facts and rules we learn etc. However the generalization abilities of our brains make us capable to generalize, move and use some experiences and knowledge to other areas, in which some similarities to the known facts and rules can be found. In order to define and develop artificial intelligence all these relations have to be explained and modelled. We know that intelligence is available in brains of living creatures, so the brain structures and their elements are the important points for research in order to reproduce them in artificial structures and elements.

Artificial Intelligence is today the branch of computer science concerned with making computers behave and solve problems like humans. This term was coined in 1956 by John McCarthy at the Massachusetts Institute of Technology (MIT). The first idea of John McCarthy was revolutional. He would like to construct machines that could work similary to the indications of human intelligence. He tried to carry out it through describing human intelligent procedures in the programming language LISP that he developed. He thought that intelligence can be reproduced in artificial systems by applying intelligent rules developed by intelligent humans. This idea allowed to reproduce intelligent behaviours only in the defined limited way, but it was a good starting point to the next research on construction of intelligent systems.

Artificial Intelligence (AI) is defined in many various ways:

• Artificial Intelligence is the capability of a machine to imitate intelligent human behavior; a branch of computer science dealing with the simulation of intelligent behavior in computers. [Merriam-Webster Dictionary]
• (Artificial) Intelligence measures an agent’s ability to achieve goals in a wide range of environments. [S. Legg and M. Hutter]
• Artificial Intelligence is the science and engineering of making intelligent machines, especially intelligent computer programs. It is related to the similar task of using computers to understand human intelligence, but AI does not have to confine itself to methods that are biologically observable. [John McCarthy]
• The goal of work in artificial intelligence is to build machines that perform tasks normally requiring human intelligence. [Nils J. Nilsson]
• Artificial Intelligence is an area of computer science that emphasizes the creation of intelligent machines that work and react like humans. [Techopedia]
• Artificial Intelligence is the theory and development of computer systems that are able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages. [Oxford Dictionaries]
• Artificial Intelligence is the simulation of human intelligence processes by machines, especially computer systems. These processes include learning (the acquisition of information and rules for using the information), reasoning (using the rules to reach approximate or definite conclusions), and self-correction. [SearchCIO]
• Artificial Intelligence is a field of knowledge including fuzzy logic, evolutionary computing, neural networks, artificial life, and robotics. Artificial intelligence is an area of computer science dealing with intelligence - intelligent behavior modelling and computer programs that simulate these behaviors. It can also be defined as the IT area involved in solving problems that cannot be efficiently transformed to algorithms. [Wikipedia.pl]
• Artificial Intelligence is the attempt to build machines which carry out tasks that would be considered to require intelligence if performed by a human.
• Artificial Intelligence should work exactly in the same way we observe in biological intelligent creatures. It uses gained knowledge and association processes to develop in a brain-like structures that can be artificial but able to reproduce real associations between data. Artificial intelligence allows to recall established associations to produce conclusions that come from the gained knowledge in the past. It can use various kinds of input receptors, even those that do not appear in nature but can be used to define classes of objects that can be associated and used in reasoning. Artificial intelligence provides generalization at various levels and is creative thanks to the ability to recall various combinations and sequences of associations in new contexts and circumstances. It is also able to create new associations that expand knowledge and form new alternative ways of thinking in various contexts and circumstances. [Adrian Horzyk]

The above definitions do not solve the problem of defining how to implement artificial intelligence on today’s computers. Also this term does not make computers to be intelligent. There is necessary to precisely define mechanisms that make intelligence to work, be real, safe, and useful.

Artificial intelligence (computational intelligence and soft-computing methods) is used in:

• expert systems,
• decision-making,
• natural language processing,
• robotics,
• machine translation,
• pattern recognition,
• clustering and classification,
• regression,
• forecasting, predicting, planning, and scheduling,
• generalizing and decision processes,
• computational creativity,
• intelligent agents and multi-agents,
• intelligent chatbots,
• games playing,
• etc.

Artificial intelligence is today imitated using various soft-computing methods. The main research in the field of intelligent computing is focussed on development of various machine learing methods using:

• artificial neural networks,
• kernel methods such as the support vector machines,
• fuzzy sets and fuzzy computation,
• genetic algorithms and gene expression algorithms,
• evolutionary programming and computation,
• rough sets, rough separability, and rough computing,
• swarm intelligence algorithms,
• mathematical optimization,
• means-ends analysis,
• statistical methods,
• k-nearest neighbor algorithm,
• decision trees,
• probabilistics and chaos theory,
• Bayes classifiers,
• uncertain numbers,
• knowledge representation and knowledge engineering,
• simulated annealing,
• hidden Markov models,
• Kalman filters,
• beam search etc.

There is a huge difference between (artificial) intelligence (being intelligent) and intelligent behaviors (behaving intelligently).
Intelligent behaviors can be simulated after the gathered facts and rules and constructed algorithms for some limited input data and defined goals. From this point of view even machines (e.g. calculator, computers, programs) can behave intelligently. Such machines are not aware of doing intelligent computations that have been programmed by intelligent programmers. It is much easier to simulate intelligent behaviors than reproduce intelligence.
On the other hand, (artificial) intelligence can produce rules and algorithms to solve new (usually similar) tasks using gained knowledge and some associative mechanisms that allow it to generalize facts and rules and then memorize or adapt the results of generalization.
Today, we can program computers to behave intelligently, but we still work on making computers to be intelligent (have their own artificial intelligence).

Artificial intelligence alike real intelligence cannot work without knowledge that have to be somehow represented, generalizable, and easy available in various contexts and circumstances. Knowledge is indispensable for implementing intelligent behaviors in machines. It automatically steers the associative processes that take place in a brain. Knowledge as well as intelligence are automatically formed in brain structures under the influence of incoming data, their combinations, and sequences.

An AI program is called an intelligent agent. The agent can perceive the state of environment through its sensors and it can efect its state with actuators. The inside function of the agent should produce the answer based on the sensed values. The environment can be: fully or partially observable, deterministic or stochastic, discrete or continuous. Artificial intelligence has also to manage uncertainty.

In computer science we usually define a sequence of actions that have to be made to achieve a demanded goal. We can distinguish:

1. Initial state (the AI agent starts)
2. Actions(s)→{a₁, a₂, a₃, ...a_n} (the actions the agent can execute, the actions are dependent on the initial state)
3. Result(s,a)→s' (we achieve a new state as a result)
4. GoalTest(s)→T/F (the true/false problem, that tell us if the problem has a solution, and the goal is able to achieve)
5. PathCost(s₁→s₂→...→s_m)→c (what is the cost of achieving goal for the given sequence of actions)

Machine learning addresses the problem of how to find a proper structure (e.g. a neural network) and its parameters to solve a problem defined using a set of training data/samples.

Training data/samples describe a given problem/task using real separate samples or their sequences. Training samples are used during learning process that takes them into account in order to find a function that will model them and allow for generalization. Generalization is one of the major beneficial features of machine learning that enables us to take advantages from learning. Generalization moves defined behaviors from training samples to new similar samples that have not been used during learning (i.e. for training an artificial intellegence system).

The AI system can be trained only if it has some adaptable elements (e.g. variables, weights, a structure of connections) that can be tuned during learning process to reflect training samples. There is usually used less number of adaptable elements than the number of training data in order to achieve generalization instead of memorization. If there is less number of adaptable elements than the number of traning data, the representation of training data have to somehow consolidate or approximate them or represent only the most representative parts of them (e.g. the most representative features, attributes, or parameters).

The example of the training samples for Iris flower classification (from ML Repository) is shown below:

This data set consists of 150 training samples that describe (model) Iris flowers using four features (parameters/attributes): leaf length, leaf width, petal length, and petal width. These flowers belong to three classes: Iris Setosa, Iris Versicolour, or Iris Virginica. The AI problem is to adjust adaptable elements of an AI systems so that possibly all these flowers will be classified correctly and also to enable correct classification of other Iris flowers that are not presented in this data set.

We distinguish supervised and unsupervised learning. The supervised learning uses teacher/expert advise to define training samples and a goal of learning. The training samples are usually defined by feature vectors and a target label (e.g. a class for classification, a demanded value for regression). The goal of such kind of machine learning is to predict the target label (e.g. a class or a value) for some new feature vectors. The unsupervised learning has no determined goal. During unsupervised learning is supposed that data will show a way to a goal (e.g. clusterization tasks). Human intelligence usually establishes goals after one’s needs and defines classes after the most frequent features of objects. Training data sets usually include defined classes or desirable output values for given input data (feature vectors).

Why artificial intelligence and human-like knowledge are so hard to implement on today’s computers?

Today’s computers are designed after the general concept of the Alan Turing machine using the John von Neumann achitecture.
The Turing machine is a hypothetical device that manipulates symbols on a strip of tape according to a table of rules. Despite its simplicity, a Turing machine can be adapted to simulate the logic of any computer algorithm, and is particularly useful in explaining the functions of a CPU inside a computer.
The John von Neumann achitecture describes a design architecture for an electronic digital computer with subdivisions of a processing unit consisting of an arithmetic logic unit and processor registers, a control unit containing an instruction register and program counter, a memory to store both data and instructions, external mass storage, and input and output mechanisms.

Today’s computers use data that are passively stored in computer memory as well as programs. Such data can do nothing and cannot actively influence the data processing, change algorithms etc. Only programs can undertake some actions and process data, but even programs are not run automatically. Moreover, the programs are developed by intelligent humans and cannot adapt themselves into new incoming data. These are the general problems that disable to develop real artificial intelligence because data cannot automatically associate and influence representations of other data. Data have to be free and able to actively influence the internal states and programs of the brain-like associative machine that can form knowledge and develop intelligence using associative structures and associative data processing.

Brains of living creatures work in a quite different way. They do not need to be externally programmed, because they can adapt to represent various data combinations, their sequences, classes, and subgroups. Brains are not Turing machines and have not John von Neumann architecture. They even have no computer-like memory to store whatever data and programs. Brains can remember some parts of data, but they are associated with other data that can be used to recall them. Such associations automatically form knowledge. The association processes are possible only in special active brain-like structures.

Associative Artificial Intelligence - is the artificial intelligence that mechanisms of working are based on associative processes, associative reasoning, associative knowledge formation and use, generalization, creativity, and associative recalling of objects, facts, and rules. It does not use the clasic data structures but the associative graph data structure (AGDS) and active graphs of associative neurons (as-neurons):

The AGDS is the graph structure that does not duplicate representation of any value or any combination of them. It also consolidates representation of the same values and associates various values according to their similarity and the given order. This structure makes all samples to be simultaneously sorted after all features. It immediately highlights all similarities, differences, duplicates, and many more... These features allow for quick reasoning without looping and searching for data in tables.

Example:
In order to find some similarities of the R8 sample to the other samples (R1, ..., R7, R9) we have to:

• search through the table a few times in many loops (computational complexity O(n)) or
• check the directly connected graph nodes of the AGDS structure (computational complexity O(1)).

We will use this associative structure in the next lectures to construct active neuroassociative graphs (AANG) - the special kind of neural networks.

Associative Artificial Intelligence will be discussed together with the knowledge formation problem because this kind of intelligence is based on knowledge, associations and a few other factors.

1. Ryszard Tadeusiewicz, Discovering neural network proprieties by means of C# programs, 2009.
2. Intelligence from the Brain Matrix point of view, and the games that support intelligence development: strategy thinking, quick recognition and classification, concentration, spacial and contextual view, and the capability for comprehending surroundings.
3. Intelligence from the Wikipedia point of view, and reflections on human, animal and even plant intelligence.
4. Horzyk, A., Artificial Associative Systems and Associative Artificial Intelligence (language: Polish), Academic Publishing House EXIT, Warsaw, 2013, postdoctoral monograph, pp. 1-280.