Attractor neural networks such as the Hopfield model can be used to model associative memory. An efficient associative memory should be able to store a large number of patterns which must all be stable. We study in detail the meaning and definition of stability of network states. We reexamine the meanings of retrieval, recognition and recall and assign precise mathematical meanings to each of these terms. We also examine the relation between them and how they relate to memory capacity of the network. We have shown earlier in this journal that orthogonalization scheme provides an effective way ✉ * E-mail: suchitra.s85@gmail.com # Contributed equally. 17 9 2020 15 9 e0238054 e0238054 24 9 2020 © 2020 Sampath, Srivastava This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Abstract Attractor neural networks such as the Hopfield model can be used to model associative memory. An efficient associative memory should be able to store a large number of patterns which must all be stable. We study in detail the meaning and definition of stability of network states. We then show that the conditions for pattern stability can be split into a necessary condition (recognition) and a sufficient one (recall). We interpret in cognitive terms the information being stored in the Hopfield model and also after it is orthogonalized. We also study the alterations in the network dynamics of the Hopfield network upon the introduction of orthogonalization, and their effects on the efficiency of the network as an associative memory. status released display-pdf yes is-olf no is-manuscript no is-preprint no is-journal-matter no is-scanned no is-retracted no Received 2020 Jan 15; Accepted 2020 Aug 10; Collection date 2020. 1 Introduction Associativity is a fundamental feature of learning and memory. When some information is learnt or memorized, it can be recalled not just when the same information is encountered again, but also by similar or partial information. The brain thus forms associations between the various information it learns and memorizes with those it encounters externally. This kind of associative memory can be modeled mathematically using some ideas from physics and mathematics which can be adapted to neuronal networks [ 1 – 6 ]. Such models of networks can help us gain insights into the mechanisms underlying learning and memory. Following this, we state the requirements for pattern stability (sec. 2.2) before exploring in detail the basins of attraction of various patterns in the network in sec. 2.3. We then study the dynamics of the network and the energy landscape in sec. 2.4. Finally, we calculate the memory capacity of the network in sec. 2.5. We then recapitulate in sec. 3 how orthogonalization is introduced in the Hopfield framework, and discuss how the basins of A comparison of the effect of correlations between the patterns on the efficiency of the Hopfield model and our modified model is presented in sec. 4. After presenting these results, we discuss their relevance to the network as an associative memory and also to cognition. 2 The Hopfield model with Hebbian learning The Hopfield model [ 1 ] is a network of N ‘neurons’ connected with each other through ‘synapses’. A neuron can take values +1 or -1, depending on whether it is firing (active) or not (inactive). Each neuron is connected to every neuron in the network except itself. Quite remarkably, there is no shifting of minima. This is apparently because the soft noise is completely eliminated due to orthogonalization. With the weights now being calculated using the { η ^ } , using Eq (9) , we can calculate the the energy of the pattern ξ ( μ ) after orthogonalization as: (12) E ( ξ ( μ ) ) = - N 2 + N 2 ( O ( p N ) ) , and depends on the value of p . As more patterns are inscribed in the network, their energies rise, as illustrated in Fig 3 (A) . Unlike in the Hopfield model, we find that the energy of a pattern is proportional to the number of patterns in the memory store, p . In the Hopfield network, as p increases, some of the ξ ’s are no longer minima. However, the energies of the attractors remains distributed around a mean (close to 0.5 N ). The average energy of the inscribed patterns is shown next to each value of p . After orthogonalization, all the ξ ’s have uniform energy for a particular value of p , and while the energy increase with the number of patterns, the ξ ’s remain minima. We should examine the implications of the changes in basin radii after orthogonalization on the memory capacity of the network and its effectiveness or quality as an associative memory. Orthogonalization brings about drastically new results for pattern stability and associative recall when it is incorporated in the framework of Hopfield model, in addition to improving the memory capacity and efficiency of the network as an associative memory. However, the catastrophic interference due to the correlations between the learnt patterns is not eliminated completely, though its effects are delayed significantly. There are a few strategies reported in literature [ 32 – 35 ] for increasing the memory capacity beyond what is set by the Hopfield model. However, the advantage of the orthogonalization scheme over the above mentioned strategies to overcome catastrophic breakdown of memory happening in Hopfield network is that it does not require any drastic changes to either the network topology or the learning rule. Experiments are needed to understand the exact biological correspondence of the dynamics of our model. Ref. [ 36 ] and models based on in vivo data, such as [ 34 , 35 ] and [ 37 ] can provide some insights into the changes in dynamics of Hopfield network on introduction of orthogonalization. Note that in reality memories are not supposed to be stable.

Net is currently the simplest and most popular neural network model of associative memory; the model allows the recall of clear target vector when cued with In mental memory, storage is one of three fundamental stages along with encoding and retrieval. Memory is the process of storing and recalling information that was previously acquired. Storing refers to the process of placing newly acquired information into memory, which is modified in the brain for easier storage. Encoding this information makes the process of retrieval easier for the brain where The multi-trace model had two key limitations: one, notion of the presence of ever-growing matrix in human memory sounds implausible; and two, computational searches for similarity against millions of traces that would be present in memory matrix to calculate similarity sounds far beyond the scope of the human recalling process. The neural network model is the ideal model in this case, as it overcomes the limitations posed by the multi-trace model and maintains the useful features of the model as well. The neural network model assumes that neurons in a neural network form a complex network with other neurons, forming a highly interconnected network; each neuron is characterized by the activation value, and the connection between two neurons is characterized by the weight value. Interaction between each neuron is characterized by the McCulloch–Pitts dynamical rule, and change of weight and connections between neurons resulting from learning is represented by the Hebbian learning rule. Anderson shows that combination of Hebbian learning rule and McCulloch–Pitts dynamical rule allow network to generate a weight matrix that can store associations between different memory patterns – such matrix is the form of memory storage for the neural network model. Major differences between the matrix of multiple traces hypothesis and the neural network model is that while new memory indicates extension of the existing matrix for th Maintenance rehearsal is mainly used for the short-term ability to recall information. Elaborate rehearsal involves the association of old with new information. === Long-term memory === In contrast to the short-term memory, long-term memory refers to the ability to hold information for a prolonged time and is possibly the most complex component of the human memory system. The Atkinson–Shiffrin model of memory (Atkinson 1968) suggests that the items stored in short-term memory moves to long-term memory through repeated practice and use. Long-term storage may be similar to learning—the process by which information that may be needed again is stored for recall on demand. Forgetting may occur when the memory fails to be recalled on later occasions. == Models == Several memory models have been proposed to account for different types of recall processes, including cued recall, free recall, and serial recall. However, to explain the recall process, the memory model must identify how an encoded memory can reside in the memory storage for a prolonged period until the memory is accessed again, during the recall process; but not all models use the terminology of short-term and long-term memory to explain memory storage; the dual-store theory and a modified version of Atkinson–Shiffrin model of memory (Atkinson 1968) uses both short-and long-term memory storage, but others do not. Because the memory matrix is constantly growing with new traces being added in, one would have to perform a parallel search through all the traces present within the memory matrix to calculate the similarity, whose result can be used to perform either associative recognition, or with probabilistic choice rule, used to perform a cued recall. While it has been claimed that human memory seems to be capable of storing a great amount of information, to the extent that some had thought an infinite amount, the presence of such ever-growing matrix within human memory sounds implausible. The neural network model assumes that neurons in a neural network form a complex network with other neurons, forming a highly Anderson shows that combination of Hebbian learning rule and McCulloch–Pitts dynamical rule allow network to generate a weight matrix that can store associations between different memory patterns – such matrix is the form of memory storage for the neural network model. Major differences between the matrix of multiple traces hypothesis and the neural network model is that while new memory indicates extension of the existing matrix for the multiple traces hypothesis, weight matrix of the neural network model does not extend; rather, the weight is said to be updated with introduction of new association between neurons. Using the weight matrix and learning/dynamic rule, neurons cued with one value can retrieve the different value that is ideally a close approximation of the desired target memory vector. As the Anderson's weight matrix between neurons will only retrieve the approximation of the target item when cued, modified version of the model was sought in order to be able to recall the exact target memory when cued. The Hopfield Net is currently the simplest and most popular neural network model of associative memory; the model allows the recall of clear target vector when cued with the part or the 'noisy' version of the vector. The weight matrix of Hopfield Net, that stores the memory, closely resembles the one used in weight matrix proposed by Anderson. Again, when new association is introduced, the weight matrix is said to be 'updated' to accommodate the introduction of new memory; it is stored until the matrix is cued by a different vector. === Dual-store memory search model === First developed by Atkinson and Shiffrin (1968), and refined by others, including Raajimakers and Shiffrin, the dual-store memory search model, now referred to as SAM or search of associative memory model, remains as one of the most influential computational models of memory.

Fractional calculus research indicates that, within the field of neural networks, fractional-order systems more accurately simulate the temporal memory effects present in the human brain. Therefore, it is worthwhile to conduct an in-depth investigation into the complex dynamics of fractional-order neural networks compared to integer-order models. In this paper, we propose a magnetically controlled, memristor-based, fractional-order chaotic system under electromagnetic radiation, utilizing the Hopfield neural network (HNN) model with four neurons as the foundation. The proposed system is solved by using the Adomain decomposition method (ADM). Then, through dynamic simulations of the internal parameters of the system, rich dynamic behaviors are found, such as chaos, quasiperiodicity, direction-controllable multi-scroll, and the emergence of analogous symmetric dynamic behaviors in the system as the radiation parameters are altered, with the order remaining constant. Finally, we implement the proposed new fractional-order HNN system on a field-programmable gate array (FPGA). The experimental results show the feasibility of the theoretical analysis. 2023 https://creativecommons.org/licenses/by/4.0/ Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license ( https://creativecommons.org/licenses/by/4.0/ ). Fractional calculus research indicates that, within the field of neural networks, fractional-order systems more accurately simulate the temporal memory effects present in the human brain. Therefore, it is worthwhile to conduct an in-depth investigation into the complex dynamics of fractional-order neural networks compared to integer-order models. In this paper, we propose a magnetically controlled, memristor-based, fractional-order chaotic system under electromagnetic radiation, utilizing the Hopfield neural network (HNN) model with four neurons as the foundation. The proposed system is solved by using the Adomain decomposition method (ADM). Then, through dynamic simulations of the internal parameters of the system, rich dynamic behaviors are found, such as chaos, quasiperiodicity, direction-controllable multi-scroll, and the emergence of analogous symmetric dynamic behaviors in the system as the radiation parameters are altered, with the order remaining constant. They can realize the basic functions of neurons by sensing electrical stimulation or excitation conduction [ 7 , 8 , 9 , 10 , 11 , 12 ]. Research has indicated that chaos phenomena can be explored within the field of neuroscience [ 13 ]. In recent years, with the advancement of research, the Hopfield neural network model has garnered significant attention in the field of neural computation [ 14 , 15 , 16 , 17 ]. The Hopfield neural network model and chaotic systems are typically composed of nonlinear equations and, when dealing with high-order nonlinear dynamics systems, insights and techniques can be drawn from methods in other fields [ 18 , 19 ]. The memristive device component possesses unique memory capabilities, serving as artificial synapses to replace native synapses within neural networks, capable of generating rich, chaotic, dynamic phenomena [ 20 ]. Bao et al. introduced a Hopfield neural network with two memristive autapse synapses per neuron, leading to the emergence of coexisting attractors in the control plane that can be shifted by toggling initial conditions [ 21 ]. Lin et al. introduced a method capable of generating n-scroll chaotic attractors by employing segmented memristors. Their approach involved investigating the electromagnetic radiation effects in a magnetized Hopfield neural network. By manipulating the synaptic weights or activation functions of the Hopfield neural network, it is possible to generate a wide range of chaotic dynamics. Consequently, the investigation of chaotic dynamics in neural networks holds significant importance. In modern times, individuals are exposed to various forms of radiation on a daily basis, and prolonged exposure to radiation can potentially lead to biological effects. Among the organs in the human body, the brain is considered to be the most sensitive to electromagnetic radiation (EMR) exposure [ 25 ]. This approach facilitates the study of the interaction between electromagnetic radiation and neural systems, opening avenues for investigating the impact of radiation on neurological processes and developing novel therapeutic interventions [ 26 , 27 , 28 ], especially on learning and memory ability. Lin et al. investigated the influence of electromagnetic radiation on the chaotic dynamics of a neural network [ 29 ]. They studied a neural network model composed of three neurons and demonstrated that an increase in the amount of external electromagnetic radiation affects the dynamic behavior of the neural network by stimulating different numbers of neurons. Wan et al. According to this algorithm, subsequent terms are numerous, but by focusing on calculating only the initial terms, the implementation complexity is greatly reduced. Therefore, it is highly suitable for FPGA implementation. 2.2. Fractional-Order HNN System Model Memristors can be employed in neuroscience to simulate biological synapses and can also be utilized to describe electromagnetic induction effects [ 53 ]. Recently, a magnetic-controlled memristor has been proposed to depict the impact of electromagnetic radiation on the HNN (Hopfield neural network) [ 30 ]. In general, certain learning and memory behaviors in the brain can be described using feedback-type neural networks such as HNN. The HNN model possesses the ability to simulate the behavior of neurons in the human brain and serves as a powerful computational tool. It exhibits high performance in handling complex problems and provides a reliable model for simulating the dynamic behaviors of brain activity.

On the behavior of some associative neural networks. Since Hopfield published his work on an associative memory model, a large number of works have studied the model from several angles and showed in particular its weaknesses, and presented ways to overcome them. Most of the proposed solutions seem to us however not biologically plausible. In this paper we present a simple statistical analysis of two networks similar to the Hopfield net, and show that the usage of positive feedback enhances the net recognizing capability without jeopardizing the stability. We also describe a layered parallel network composed of modules, each module being a modified Hopfield net. We finally present computer simulation results to support our analytical findings. The most important principles of this network are supported by data from the world of neurobiology. Published in Biological cybernetics (1988)

The content addressable associative memory based on the Hopfield neural network model can be used directly to store and retrieve information with robustness and error correction capability. The Hopfield associative memory can successfully recall data stored only when the stored data satisfy some stringent conditions. To overcome these limitations of the Hopfield model, some modifications have been proposed. In this paper we analyze a limitation of the Hopfield model, present a modification of the Hopfield model, and at last give its numerical simulation.