Neuro-inspired Information Processing

Alain Cappy

• Publisher: Wiley-ISTE
• Year: 2020
• Print ISBN: 9781786304728
• eBook ISBN: 9781119721819
• Edition: 1
• Topic: Technology & Engineering
• Subtopic: Electrical Engineering & Telecommunications
• Index: Technology & Engineering

With the end of Moore's law and the emergence of new application needs such as those of the Internet of Things (IoT) or artificial intelligence (AI), neuro-inspired, or neuromorphic, information processing is attracting more and more attention from the scientific community. Its principle is to emulate in a simplified way the formidable machine to process information which is the brain, with neurons and artificial synapses organized in network. These networks can be software – and therefore implemented in the form of a computer program – but also hardware and produced by nanoelectronic circuits. The ?material? path allows very low energy consumption, and the possibility of faithfully reproducing the shape and dynamics of the action potentials of living neurons (biomimetic approach) or even being up to a thousand times faster (high frequency approach). This path is promising and welcomed by the major manufacturers of nanoelectronics, as circuits can now today integrate several million neurons and artificial synapses.

1
Information Processing

1.1. Background

Since the beginning, humankind has forever created and used various techniques to process, transmit and memorize information, a vast domain summed up by the term “information processing”. This concept is very different from that of intelligence, and the reader should take care not to confuse these two ideas. This is a conflation commonly witnessed today owing to the buzz around the idea of artificial intelligence, which often only covers Big Data processing. Misuse of this term has seen the concepts of “intelligence” and “processing power” come to be erroneously employed interchangeably.

Intelligence is a much more complex matter that is beyond the scope of this book, referring instead to adaptation and imagination capacity. Moreover, Einstein is credited with the maxim: “The true sign of intelligence is not knowledge, but imagination.”

Generally speaking, information processing can be broken down into several phases, set out in Figure 1.1.

Let us elaborate on this figure a little. One might say that for a piece of information (a sign, sound, color, etc.) existing in the real world, the first operation necessary before any processing is its acquisition through encoding.

Figure 1.1. Block diagram of information processing systems. For a color version of this figure, see www.iste.co.uk/cappy/neuro.zip

Our senses, such as our vision and hearing, use sensors, in this case the retina and the cochlea, whose role it is to encode visual and auditory information so that they can be processed by the brain. If this same visual and auditory information from the real world is captured by a camera and a microphone, the encoding will be different and may be analog or binary. Once encoded, the information can be processed. Processing machines and their internal organization can be very diverse, ranging from the brain to computers, but all possess a device enabling information memorization and communication, whether this be within the machine itself, or with a similar machine. After processing, the information, which exists in the machine’s encoding system, will need to be decoded in order for it to be interpretable in the real world. Decoding is, for example, the role of the computer screen displaying a computation result.

Let us briefly develop on these basic concepts.

1.1.1. Encoding

All processing requires a prior information encoding operation. To illustrate this concept, let us imagine that the information in question is the temperature, T, of a physical environment.

Figure 1.2. Different information encoding types

Encoding can be analog (Figure 1.2a), in which case, the “temperature” magnitude will be represented after encoding by a continuous value¹ x (e.g. voltage or current), such that:

[1.1]

where

is the encoding transfer function. If, for example,

is a linear function, we will have x = α.T + β, where α and β are real constants.

When encoding is analog, the information processing must also be analog, in which case we speak of analog computers.

Another way to process this information is to code it in binary. In this case, we choose a pitch ΔT, and T will be represented by a number, M, of N bits such that:

[1.2]

One of the N bits (Figure 1.2b) will only be able to take on two values, arbitrarily noted 0 and 1. Binary coding offers numerous advantages: it enables the use of Boolean algebra and the set of associated algorithms for processing encoded information, and can easily be implemented with physical systems in two entirely distinct states. The switch, in particular, is either open (state 0) or closed (state 1).

Other encoding methods also exist. In the brain, for example, the organ focused on in this book, information is both time and frequency encoded by electrical impulses k...

Table of contents

• Cover
• Table of Contents
• Acknowledgments
• Introduction
• 1 Information Processing
• 2 Information Processing in the Living
• 3 Neurons and Synapses
• 4 Artificial Neural Networks
• References
• Index
• End User License Agreement