Plate detail (12 inches) of semiconductor material -
© Peellden / Wikipedia CC BY-SA
Artificial intelligences can now emulate the functionalities of the cerebral cortex thanks to “deep learning”, according to a study published by our partner The Conversation.
But the largest network of artificial neurons and synapses in existence requires a computing power of several megawatts, compared to the few watts consumed by a human brain doing the same task.
The analysis of this paradox was carried out by Alain Cappy, professor emeritus in electronics at the University of Lille.
The brain remains the king of computers.
The most sophisticated machines inspired by it, called "neuromorphic", today include up to 100 million neurons, or as many as the brain of a small mammal.
These networks of artificial neurons and synapses are the basis of artificial intelligence.
They can be emulated in two ways: either with computer simulations, or with electronic components reproducing biological neurons and synapses, assembled into “neuroprocessors”.
These software and hardware approaches are now compatible, which suggests drastic developments in the field of AI.
How does our brain work?
Neurons, synapses, networks
The cortex is the outer layer of the brain.
A few millimeters thick and the surface of a napkin, it contains more than 10 billion neurons that process information in the form of electrical impulses called “action potentials” or “spikes”.
The connection point between a neuron which emits a
spike
(the preneuron) and the neuron which receives it (the postneuron) is the synapse.
Each neuron is connected by synapses to around 10,000 other neurons: the connectivity of such a network, the connectome, is therefore prodigious.
The function of neurons is fixed: it consists in summing the signals coming from the synapses, and if this sum reaches a threshold, in generating an action potential or
spike
which will propagate in the axon.
It is remarkable to note that part of the processing is analog (the sum of synaptic signals is
continuous
) while the other is binary (the neuronal response is either the generation of a
spike
or nothing).
Thus the neuron can be considered as an analog computer associated with a digital communication system.
Unlike neurons, synapses are plastic, that is to say they can modulate the intensity of the signal transmitted to the postneuron, and have a “memory” effect, because the state of a synapse can be preserved. in time.
Network of biological neurons and circulation of the action potential or "spike" © Alain Cappy
Anatomically, the cortex divides into approximately one million cortical columns, which are networks of neurons all with the same interconnecting architecture.
Cortical columns can be thought of as elementary processors, of which neurons are the basic devices and synapses are memory.
From a functional point of view, the cortical columns form a hierarchical network comprising connections going from the bottom (the sensory sensors) to the top, which allows the abstractions, but also from the top to the bottom, to allow the predictions: the processors in our brain work both ways.
The main challenge of AI is to emulate the functionalities of the cortex with networks of neurons and artificial synapses.
This idea is not new, but it has taken off in recent years with
deep learning
, or “deep learning”.
Use software to simulate neural and synapse networks
The software approach aims to simulate neural and synapse networks with a standard computer.
It has three ingredients: mathematical models of neurons and synapses, an interconnection architecture of neurons, and a learning rule that makes it possible to modify "synaptic weights".
Mathematical models of neurons range from the simplest to the most realistic (biologically), but simplicity is required to simulate large networks - several thousand or even millions of neurons - to limit computation time.
The architecture of artificial neural and synapse networks generally includes an input "layer", containing the "sensory neurons", and an output layer, the results.
Between the two, an intermediary network which can take two main forms: "feedforward" or "recurrent".
Architecture of a “forward” neural network, or feedforward. The parameters of such a network are the number of layers, the number of neurons per layer, and the rule of interconnection from one layer to the next. For a given task, these parameters are generally chosen empirically and often oversized. Note that the number of layers can be very high: more than 150 for Microsoft's ResNet for example © Alain Cappy
In a feedforward network, information is transferred from one "layer" to the next, without a feedback loop to the previous layers.
On the contrary, in recurrent networks, connections may exist from one layer
N
to the previous
N-1
,
N-2
, etc.
Consequently, the state of a neuron at the instant
t
depends both on the input data at the instant
t
, but also on the state of the other neurons at the instant
t-Δt.
, which significantly complicates the learning process.
“Recurrent” neural networks contain feedback loops. In recurrent networks, the “time” variable is an essential parameter © Alain Cappy
The learning aims to determine the weight of each synapse, that is, the intensity with which the
spike
from a preneuron is transmitted to the postneuron, so that the network can meet a defined goal.
There are two main types of learning: supervised when a “teacher” (or “master”) knows the expected result for each entry and unsupervised when such a “master” is absent.
In the case of supervised learning, it is the comparison between the result obtained for an entry and that of the “master” which makes it possible to adjust the synaptic weights.
In the case of unsupervised learning, it is a rule like Hebb's famous rule which allows the synaptic weights to change during the various tests.
Building hardware artificial neural and synapse networks
The hardware approach is to design and manufacture neuroprocessors that emulate neurons, synapses and interconnections.
The most advanced technology is based on the industry of standard semiconductors (known as CMOS), used in our computers, tablets and other smartphones.
It is the only sector currently sufficiently mature to manufacture circuits comprising several thousand or millions of neurons and synapses capable of performing the complex tasks required by AI, but technologies based on new devices are also proposed, for example in spintronics or using memristors.
Like biological networks, networks of hardware artificial neurons and synapses often combine an analog part for the integration of synaptic signals and a digital part for communications and storage of synaptic weights.
This type of mixed approach is used in the most advanced technologies, such as chips from the European Human Brain project, from Intel or TrueNorth from IBM.
The TrueNorth chip, for example, combines one million neurons and 256 million programmable synapses, divided into 4,096 neuromorphic hearts - similar to the cortical columns of living things - linked together by a communication network.
The power consumed by the TrueNorth chip is 20 mW per cm2, while that of a conventional microprocessor is 50 to 100 W per cm2, which is an energy gain greater than 1000 (the usual is to consider the "surface density power ”, because not all chips have the same surface).
Will the future be hardware or software?
Software artificial neural and synapse networks provide an elegant solution to many problems, especially in the fields of image and sound processing and more recently of text generation.
But learning recurrent artificial neural and synapse networks remains an example of major difficulty, whether through supervised or unsupervised methods.
Another problem is that the computational power required becomes considerable for the large artificial neural and synapse networks required to solve complex problems.
The GPT-3 program feeds on data that does not need to be labeled.
For example, the impressive results of the conversational program "GPT-3" are based on the largest network of artificial neurons and synapses ever built.
It has 175 billion synapses, and requires a considerable computing power made up of 295,000 processors which consume an electrical power of several megawatts, that is to say equivalent to the power consumed by a city of several thousand inhabitants.
This value is to be compared to the few watts consumed by a human brain which performs the same task!
The material approach and the neuroprocessors are much more efficient on the energy level, but they suffer from a major difficulty: the passage to scale, that is to say the manufacture of several million or billion neurons and synapses and their interconnection network.
"Artificial intelligence" dossier
In the future, and to the extent that neuroprocessors use the same CMOS technological path as usual processors, the co-integration of software and hardware approaches may pave the way for a new way of conceiving information processing and therefore efficient and energy efficient AI.
High Tech
Can an AI (really) guess your health from listening to you speak?
Science
Future robots will be more "softer", therefore less dangerous for humans
This analysis was written by Alain Cappy, professor emeritus in electronics at the University of Lille. The original article was published on The Conversation website.
Computer science
Video
Artificial intelligence
High Tech
VISIONS
Brain