Structure

Most researchers today would agree that artificial neural networks are quite different from the brain in terms of structure. Like the brain, however, a neural net is a massively parallel collection of small and simple processing units where the interconnections form a large part of the network's intelligence; however, in terms of scale a brain is massively larger than a neural network, the units used in a neural network are typically far simpler than neurons, and the learning processes of the brain (whilst unknown) are almost certainly distinct from those of neural networks.

Models

A typical feedforward neural network will consist of a set of nodes. Some of these are designated input nodes, some output nodes, and those in between hidden nodes. There are also connections between the neurons, with a number referred to as a weight associated with each connection. When the network is in operation, a value will be applied to each input node - the values being fed in by a human operator, or from environmental sensors, or perhaps from some other program. Each node then passes its given value to the connections leading out from it, and on each connection the value is multiplied by the weight associated with that connection. Each node in the next layer then receives a value which is the sum of the values produced by the connections leading into it, and in each node a simple computation is performed on the value - a sigmoid function is typical. This process is then repeated, with the results being passed through subsequent layers of nodes until the output nodes are reached. Early models (circa 1970) had a fixed number of layers. More recently, genetic algorithms are used to evolve the neural structure. See: Neuroevolution.

Calculations

Alternative calculation models in neural networks include models with loops, where some kind of time delay process must be used, and "winner takes all" models, where the neuron with the highest value from the calculation fires and takes a value 1, and all other neurons take the value 0.

It should be noted that the sigmoid curve curve is used as a transfer function because it has the effect of "squashing" the inputs into the range [0,1]. Other functions with similar features can be used, most commonly tanh() which has an output range of [-1,1]. The sigmoid function has the additional benefit of having an extremely simple derivative function for backpropagating errors through a feed-forward neural network.

Typically the weights in a neural network are initially set to small random values; this represents the network knowing nothing. As the training process proceeds, these weights will converge to values allowing them to perform a useful computation. Thus it can be said that the neural network commences knowing nothing and moves on to gain some real knowledge.

Usefulness

Neural networks are particularly useful for dealing with bounded real-valued data, where a real-valued output is desired; in this way neural networks will perform classification by degrees, and are capable of expressing values equivalent to "not sure". If the neural network is trained using the cross-entropy error function (see Bishop's book) and if the neural network output has a sigmoidal non-linear, then the outputs will be estimates of the true posterior probability of a class.

Real life applications

In real life applications, neural networks perform particularly well on the following common tasks:

    * Function approximation
    * Time series prediction
    * Classification
    * Pattern recognition

Other kinds of neural networks, in particular Continuous-Time Recurrent Neural Networks (CTRNN), are used in conjunction with genetic algorithms(GAs) to produce robot controllers. The genome is then constitued of the networks parameters and the fitness of a network is the adequacy of the behaviour exhibited by the controlled robot (or often by a simulation of this behaviour).