Learning and Hopfield Networks

Learning and Hopfield Networks

Introduction

Learning involves the formation patterns of neural wiring that are very useful irrespective of presence or absence of external feedback from the supervisor. For instance, there are neural wiring patterns in both absence and presence of external feedback from the environment or an instructor. Thus, neural networks (both artificial and biological) that facilitate learning both in supervised and unsupervised means are usually very essential in enabling extraction of useful relationships from input/sensory data. However, the patterns extracted may be invariant patterns in key variants and the data that allow differentiation of important input classes in the system (McClellan et al., 1995).  Either way, these concepts are learned by the system on itself. Therefore, learning systems in the human brain-mind system takes place through the biological neural network and the Hopfield networks as models plays a very important role for actual human learning where the sequence of items learned is also included (Hopfield, 1982). The Hopfield network resonates with the emphasis of Chomsky on the role of word sequence and syntax in the process of learning language (Chomsky, 2009).

Hopfield network is one of Artificial Neural Networks (ANN) which is involved in processing of information paradigm whose inspiration originates from the by the way in which processing of information takes place in the brain (Squire & Kandel, 1999). ANNs are mathematical models involved in the emulation of some of properties of biological nervous systems observed and drawn on the adaptive biological learning analogies. This implies that Hopfield network is not only concerned with emulation of the learning systems in the human brain system, but also an analytical tool (McClellan et al., 1995). Therefore, Hopfield network as an ANN ii is usually composed of an extensive number of processing elements that are highly interconnected that are analogous to biological neurons and interconnected together by connections that are analogous to synapses (Burgess & O’Keefe, 1994). However, Hopfield networks not only share the brain-mind system learning characteristics, but they are also important analytic tools.

The Hopfield Network as an Analytical Tool

The Hopfield network has the possibility of acting as an analytical tool since it is represented as nodes in the network that represents extensive simplifications of real neurons, and they usually exist in either firing state or not firing state (Hopfield, 1982). In addition, all the nodes in a Hopfield network are connected to each other with some strength which allows it to execute its functions in an effective manner. Apart from the Hopfield network deriving meaning from sets or subsets of data for the purpose of facilitating learning process, it also make sure that relationships from the same sets or subsets of data which means they provides an aspect of data analytics (Squire & Kandel, 1999). This implies that apart from Hopfield network playing a crucial in facilitating brain-mind system learning, it is also a very useful analytical tool which through the relationships it derives from data serves as a way of ensuring the meaning of data is achieved. As an effective analytical tool, Hopfield network at any time will make sure that the interconnected nodes will change their state (i.e. stop or start firing) on the basis of received inputs from the other nodes (McClellan et al., 1995).

This implies that Hopfield can also be used as a powerful analytical tool to create relationships between received inputs or solve combinatorial problem (Hopfield, 1982). Furthermore, the analytical tool aspect of Hopfield involves certifying the accuracy of algorithm. Rizzuto and Kahana (2001) showed that the neural network model, especially the Hopfield network model is essential in the process of accounting for repetition on recall accuracy through incorporation of an algorithm that is based on probabilistic-learning and analysis. It is also reiterated that during the retrieval process which is common during analysis, no learning occurs thereby making the network weights to remain fixed in addition to showing the capability of the Hopfield network model to switch to a recall stage from a learning stage (McClellan et al., 1995).

Moreover, in the attempts of using Hopfield network model as an analytical tool, combinatorial optimization it highly prioritized especially through correct modelling of the problem (Hopfield, 1982). Therefore, the Hopfield network model has the ability to give some solution and some minimum but in most cases it is not able to find the optimal solution to a problem even though the analysis provided is often sufficient. This implies that Hopfield network is generally used as a “black box” for the calculation of some output that result from a particular self-organization because of the network (Burgess & O’Keefe, 1994). Therefore, due to the ability of Hopfield network model to be used as an analytical tool, it is then possible to use it for combinatorial problem. This is mainly because, when a problem is writable in the energy function of an isomorphic form, then it is possible to find the function’s local minimal using the Hopfield network model, but there is no guarantee that the solution provided will be optimal (Squire & Kandel, 1999).

For example, the use of Hopfield network model as an analytical tool has been applied in the travelling salesman problem. This is mainly because this is an NP-hard problem in the analyses involved in combinatorial optimization (Burgess & O’Keefe, 1994). For instance, since travelling salesmen travel extensively then, given a list of cities alongside their distances in pair, the Hopfield network model can be used to find a tour that will be shortest of all others, but enables each city to be visited by the salesman (Rolls & Treves, 1998). In this scenario, the tour path passes through n number of cities, and each of those cities should only be visited by the salesman once prior to returning to his or her original point of departure as well as ensuring that the distance covered is minimal.  However, Hopfield network has been extensively used to resolve this combinatorial problem after it has been expressed in terms of energy function, but absolute optimality is not always guaranteed (Rolls & Treves, 1998). Irrespective of this shortcoming, Hopfield network model remains a very useful analytical tool since when unit is regarded as a small processor, Hopfield neural network can be used to offer high power of parallelism and computation. This is achievable because of the inherent analytical capability of the Hopfield network which can undergo asynchronous updating (Squire & Kandel, 1999).

Conclusion

Since the Hopfield network was published in 1982 by John Hopfield, it became a breakthrough in neural network as a result of providing significant dynamism to the research on neural networks, particularly on their analytical capacity. Much has so far been discovered and these findings are nowadays used to refine the analytical capabilities of artificial neural systems, especially in the field of computer science.

 

References

Burgess, M. R. N. & O’Keefe, J. (1994). A model of hippocampal function, Neural Networks, 7, pp. 1065-1083.

Chomsky, N. (2009). Mysteries of Nature: How Deeply Hidden? Journal of Philosophy, 106 (4): 167–200.

Hopfield, J. (1982). Neural networks and physical systems with emergent collective computational abilities, Proceedings of the National Academy of Sciences, 79, pp. 554-2558.

McClellan, J., McNaughton, B. & O’Reilly, R.C. (1995). Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory, Tech. Rep. PDP.CNS.94.1, Carnegia Mellon University. March, 1994.

O’Reilly, R. C. & Munakata, Y. (2000). Computational Explorations in Cognitive Neuroscience. Cambridge, MA: The MIT Press.

Rizzuto, D. S. & Kahana, M. J. (2001). An auto-associative neural network model of paired-associate learning. Neural Computation, 13, 2075-2092.

Rolls, E. & Treves, A. (1998). Neural Networks and Brain Function. New York, NY: Oxford University Press.

Squire, L. & Kandel, E. (1999). Memory: From Mind to Molecules. New York, NY: Henry Holt and Company.

 

Use the order calculator below and get started! Contact our live support team for any assistance or inquiry.

[order_calculator]