The Biological Plausibility of Neural Networks

 

W.Williams

 

"The networks to be explored attempt to simulate natural neurons with artificial units...Each unit receives input signals from other units via "synaptic" connections...the "axonal" end branches from other units all make connections directly to the "cell body" of the receiving unit" (Churchland 1989 as cited in Berkeley 1997).

 

The goals of connectionism are to both investigate the workings of the human brain and also replicate some of its basic functions for clinical tasks such as voice/face recognition. The appeal of neural networks is evident within the high level of research and rapid growth and enthusiasm of the subject. Bachtel and Abrahamsen (1991 as cited in Berkeley 1997) explain "network models were attractive because they provided a neural like architecture for cognitive modelling". Artificial neural networks consist of (neural) units, which are the equivalent of single neurons. The units are divided into layers, most commonly consisting of an input layer which are equivalent to sensory neurons, and an output layer, equivalent to motor neurons. This simple two layer structure would be analogous of a simple reflex system in the body, and so a number of hidden layers lie between the input and output units, equivalent to all of the interconnecting nerves of the body, or most significantly, brain.

 

The similarities between the biological and artificial neural networks are apparent at a metaphorical level, however it becomes a far more subjective area the further the comparison is taken. Rumelhart (1989 as cited in  Berkeley 1997) describes connectionist processing as something close to an abstract neuron.

 

The ultra-structure of a neuron consists primarily of dendrites, a nucleus encapsulating soma, axon, and synaptic junctions connecting the neuron to many others. The physical structure of a neuron is represented by a single unit in an artificial neural network, microcosmically illustrating the simplification of connectionism as a metaphor to a far more complex system. Winlow (1990 as cited in Berkeley) highlights the common belief of neuroscientists that there is no such thing as a typical neuron. Churchland and Sejnowski (1994) also draw attention to the fact that there are 12 different kinds of neurons in the neocortex alone. With this high level of variability, a simple representation of a neuron may contain some functional characteristics of neurones, but may also omit some subtle but possibly highly relevant differences. The features employed within the "abstract representation" bare no obvious significance, a concern supported by the lack of justification on the part of neuroscientists for the neuronal features not included within the network model. So it is apparent that one biological implausibility of network models is the unrealistic homogeneity, which fails to reflect the high complexity of a biological system. It is also important to note the significance of different neurons within biological systems, with regards to the patterns of inter-connectionism. Sereno (1988 as cited in Churchland and Sejnowski) argues that in the brain, "most connections are between, not within, cell classes".

 

The threshold level of an action potential is represented in a unit by a bias, a set but variable value which influences the activation of one unit to the next. Within neurons however the threshold is set, and lacks the plasticity available with a bias figure. The significance of the inter-unit influences will perhaps be illustrated more effectively when considered at a synaptic level.

 

The pre-synaptic bouton of an inter-neuronal junction contains a highly specific neuro-transmitter of which is responsible for the transmission of an action potential across the synaptic cleft. Within artificial neural networks, the synapses are represented by weights, numerical values attached to each unit. The values maybe a negative or a positive figure, reflecting the inhibitory or excitatory nature of the neuron. The unit receiving input signals performs a variable function incorporating the values, as well as a possible bias figure as mentioned earlier. The function is commonly a summation of the signals, which may be translated into an either/or output of 1 or 0. It may also consist of a more complicated sine wave function, whereby an exponential output is computed. Biologically, the axonal and dendritic structures are a definite part of the whole neuron, with connectionist models however they are represented by a value attached but not part of the unit.

 

A further difference can be seen by the organisation of the connections within the network model. Connectionist networks exhibit a highly parallel format of hierarchy, whereby every unit is connected to every unit in the next and previous layers as illustrated in figure (a). However, the brain organises neurons in a far more unpredictable fashion. Stevens (1989 as cited in Churchland and Sejnowski 1994 pp51) highlight the difference as follows:

 

"Not everything is connected to everything else. Each cortical neuron is connected to a roughly constant number of neurons, irrespective of brain size, namely about 3% of the neurons underlying the surrounding square millimetres of cortex. Hence...cortical neurons are actually rather sparsely connected..."

 

This illustrates the high biological implausibility of connectionism with regards to organisation. A further important difference lies not in the number of connections but where they are made. Churchland and Sejnowski (1994 pp51) also note that "Forward projections to one area are generally matched by recurrent projections back to the area of origin". This is contradictory to the forward flowing of a process through an artificial neural network. Also, the level of influence a neuron has on the next connected layer is thought to be far weaker than is apparent within an artificial system. On average, the action potential emitted by a neuron cell contributes to only 1% - 5% of the firing threshold. Within connectionism however the level of influence is directly upon the number of units within a layer.

 

The nature of the action potential is a relevant feature not reflected within a unit. Units use the activation of a specifically attributed numerical value. Neurons however transmit the signal by the production of spiked pulses of an electrical voltage. The firing patterns also differ according to specific neurons. Some are a direct result of their recent firing history, some feature oscillatory patterns. Also, action potentials maybe transmitted either electrically or chemically. The properties of such a method and it's variances are weakly translated to the simpler imitation, although possibly related to the summation function within a unit there is no relevance or order as exhibited within the biological system. As it is highly likely that the specific firing nature of neurons are directly related to their function and contribution to a given process, the biological validity of artificial neural networks is made more implausible.

 

As well as the type and strength of an action potential, further biological differences can be made with such significantly effecting features as the diameter and length of axons. They respectively influence the speed of a signal and the cut-off level of a frequency, so that a signal may reach the synapse to one neuron or another depending on the axon length and signal strength. This duality and specificity is another feature not represented within artificial systems.

 

Once the individual structures and connections of neurons have been considered, the neurological processes involved are unsurprisingly found to be also substantially different. However, whilst the functional qualities of neural networks are far more complex than their artificial counterpart, some vast simplifications are found biologically with regards to cognitive processes as a whole. Once electronically constructed, neural networks are trained through a very lengthy procedure, dependant on the complexity of the task it is to perform. The weights attached to the units are initially set at random levels, and through a process of back propagation, the weights are favourably altered. This process is known as Hebbian learning, and carries the phrase 'neurons that fire together, wire together' (Hebb). For example, consider a system whereby a network is required to distinguish between a monkey or a dog. Pictures of dogs and monkeys would be inputted to a neural network organised as a perceptron (a system whereby light meters would register pixels of the image and give either a 1/0 value dependant on whether the pixel was dark or light, or a -1 to 1 value to indicate a shade of grey). The input layer would therefore consist of a reasonably large number of units depending on the number of pixels utilised. The output layer would consist of only one unit, whose value would indicate the specie (e.g. 0 = monkey, 1 = dog). Depending on whether the task was linearly separable or not, a number of hidden layers may be used which wouldn't exceed the number of input units. A number of training sets would be used, by showing the perceptron images of dogs and monkeys. If the perceptron performed the task incorrectly as would be expected from random weights, the weights would be altered by a set increment in the direction that is required for the task to be performed correctly. This method can require a great many number of training epochs, sometimes requiring months of training for a standard recognition task (depending of course on the complexity of the task and network). Even when trained, the network model may exhibit an unreliable level of generalisation, whereby a new input not used in training would possibly not be correctly classified. The back propagation method however is immediately biologically implausible (Quinlan 1991), as natural systems can learn far more rapidly and consistently. Sometimes, for example never eating a food once found to be distasteful only requires one instance as an input to be 'trained' sufficiently.

 

The importance of specific biological plausibility however can be questioned. The efficiency of an imitation model does not have to depend on the processes involved, but on the relationship between the input and output. Consider a multiplication (Berkeley 1997). A human tackles mental multiplication through a process known as the 'classical multiplication algorithm'. A calculator however performs the same task but using the entirely different method of 'multiplication a la Rosse', more applicable to the digital structure of it's circuitry. Although performing the same task, one wouldn't consider examining a calculator to investigate how the mind works, whilst a calculator performs the task far more rapidly, efficiently and reliably than the average human. Similarly, the emphasis on biological plausibility may be irrelevant when considering neural network models. The differences would be important if a replication of the human brain was to be attempted, incorporating all of it's faults and specialities with only simple units. This offers relevance with regards to artificial intelligence, but even then clinical intelligence would be a more viable product of networking than human intelligence. When performing specific tasks, the integrity of artificial neural networks lies in their clinical processing and reliable generalisation when trained sufficiently. The implications of such networks are already being utilised in many fields of technology, from product fault recognition in factories, to employer recognition in security cameras, to handwriting recognition in lap top computers.

 

Artificial neural networks offer functional processes analogous to cognitive processes. However the high complexity of the brain's organised network of neurons, neuronal structure and complex connectionism is seemingly far too complicated to be even partially replicated, as the whole system is so specialised. However, the individual functions of some simple processes maybe clinically captured within a network of units, through the use of unit values, weights and biases. Although the network will be trained in a fashion also different to that of a biological system, through numerous training epochs the pattern between input and output may be sufficiently competent to perform a task. Whilst offering a highly simplistic model of a cognitive process, the comparison to biological networks is insignificant when considering the vast implications and applications of connectionism. As research and technology in this area continues to grow at an exponential rate, the limitations of artificial networks are a feature which also unlike biological networks, have yet to be met.

 

References

 

Berkeley, I.S.N (1998) Connectionism Reconsidered: Minds, Machines and Models. University of Southwestern Louisiana.

 

Berkeley, I.S.N (1997) Some Myths of Connectionism. University of Southwestern Louisiana.

 

Bear, M.F, Connors, B.W, Paradiso, M.A (1996) Neuroscience: Exploring the Brain. Williams and Wilkins.

 

Churchland P.S. and Sejnowski T.J (1994) The Computational Brain. MIT press.

 

Quinlan, P. (1991) Connectionism and Psychology. Harvester Wheatsheaf.