Talk:Hopfield network

Incomplete descriptions of Running

According to the description under the section Running, a node must be picked, after which the behavior is deterministic. This is a rather incomplete description. How is the behavior of the node defined after it is picked? I suppose it is updating or replacing the value s_i with it's activation a_i (according to the definition given earlier), but it would be better if this is stated explicitly. —Preceding unsigned comment added by 140.78.124.18 (talk) 17:06, 29 June 2009 (UTC)[reply]

Further Correction - symmetric weights

My error - I misread the context of his statement. The condition of symmetric weights guarantees that following the update rule makes energy a monotonically decreasing function, which guarantees convergence to local minima, however, non-symmetric weights do not seem to impare the use of the network as a content-addressable memory system.

Correction - connections need not be symmetric!

If you refer to the origina Hopfield paper ( citied at the bottom of the page ) he discusses the performance of networks with the "special case" of symmetric weights, but says that the network performs just as well with non-symmetric weights. Specifically he says: "The flow in phase space produced by this model algorithm has the properties necessary for a content-addressable memory whether or not Tij is symmetric" (Hopfield, 1982, p. 2556)

That's right. He says in the 1982 paper (and repeats in the ones from 1984 and 1986) that the weights should be more or less symmetric in order to converge. --Ben ^T/_C 14:27, 5 July 2007 (UTC)[reply]

Connection between Hopfield Net and Ising model?

Hello!

I had some classes this week which involved the definitions of Hopfield networks and Ising model, and came here to look for further information/links.

There is a link in this article to Ising model, but nothing is written in the article body that explains the connections between the two concepts, maybe someone could fill that gap in?

(I'll try after I've studied enough to understand the connection myself).

Cheers

The Ising model is a model of ferromagnetism. Atoms are bipolar (i.e. either positive or negative) and they have connections and local interactions of atoms can lead to some state transitions on a global level. They are the theoretical foundation of Hopfield Networks and Hopfield specifically mentions them and changes the atoms to McCulloch-Pitts neurons, i.e. he gives them a threshold. --Ben ^T/_C 14:33, 5 July 2007 (UTC)[reply]

Definitions?

The relation between the a[i]'s and the s[i]'s is not clear. Are the a[i]'s just the updated values of the s[i]'s? In that case, why not call them both s[i]?

Another terminological matter: The article says

Hopfield nets can either have units that take on values of 1 or -1, or units that take on values of 1 or 0.

and goes on to give the updating rules in the two cases. This seems like to much attention to a trivial matter of scaling. I would suggest choosing one convention or the other for the article and then mentioning that the other convention is also used. --Macrakis 16:21, 15 August 2006 (UTC)[reply]

I also don't find it that important, whether they are ${0,1}^{n}$ or ${-1,1}^{n}$ . But I find it important that units can be also continuous. Bipolar units are only one particular case studied in Hopfield's papers. --Ben ^T/_C 14:37, 5 July 2007 (UTC)[reply]

Energy formula

Currently energy is written as:

E=-{\frac {1}{2}}\sum _{i<j}{w_{ij}{s_{i}}{s_{j}}}+\sum _{i}{\theta _{i}\ s_{i}}

I feel this is incorrect. Either removing 1/2

E=-\sum _{i<j}{w_{ij}{s_{i}}{s_{j}}}+\sum _{i}{\theta _{i}\ s_{i}}

or summing over all $i$ and $j$

E=-{\frac {1}{2}}\sum _{i,j}{w_{ij}{s_{i}}{s_{j}}}+\sum _{i}{\theta _{i}\ s_{i}}

would fix the problem. But I'm not so confident to modify the main text. I'd appreciate if somebody could check it. -- i agree, and i've changed it, (before looking here) I T.A a neural networks course... you can easily see this be derivating E w.r.t S_j to get h_j

right. --Ben ^T/_C 14:38, 5 July 2007 (UTC)[reply]

energy

I have an argument on energy function. Sometimes threshold is a more complicated function and we cannot easily incorporate it into Energy function. I mean as I have seen in "Associative memories - the Hopfield net", it should not contain this term:

\sum _{i}{\theta _{i}\ s_{i}}

Am I right? ±±±±±±± —Preceding unsigned comment added by 131.123.28.88 (talk) 19:57, 20 February 2008 (UTC)[reply]

Name of article

Would it not be more encyclopedic for this article to be entitled Hopfield _Network_, rather than Hopfield _Net_? The reasons should be obvious. Opinions?65.183.135.40 (talk) 06:21, 6 March 2008 (UTC)[reply]

Associative memory: Terminology and cross-refs

I am confused and the Wikipedia is currently not in a very helpful state regarding this:

Is "Content-addressable memory" synonymous to "associative memory" (as indicated here) or is it a term specifically describing a kind of memory hardware (as indicated in the article by that title)? Is a Hopfield network also an "auto-associative memory"?

For right or wrong, I felt an urge to make the following modifications:

1) I modified the crossref to "Content-addressable memory" to point to the more general "associative memory" (disambig); this I did because the article on CAM is too specific and (for the time being) not appropriate to reference from here. Depending on the correct answer to my confusion question, perhaps it is the CAM article that should be generalised to make it compatible with the ANN context?

2) I also added See also: Associative memory and Auto-associative memory.

195.60.183.2 (talk) 18:19, 17 July 2008 (UTC)[reply]

Training and Limitations

As it stands right now, the article doesn't say how one can train Hopfield nets to make particular inputs become local minima. It also doesn't say whether you can have two Hopfield nets with the same local minima but different "basins of attraction".

Also, I assume there is some limit to the number of minima that a network can be trained to (I would guess something like the logarithm of the number of nodes, or something similar) but the article doesn't say anything about this. If there is in fact no limit, it would be good to mention that.

120.18.115.209 (talk) 06:39, 15 October 2009 (UTC)[reply]

Binary treshold unit = Perceptron

Perhaps the Binary treshold unit section can be merged with perceptron —Preceding unsigned comment added by StookWagen (talk • contribs) 10:39, 2 December 2009 (UTC)[reply]

Picture

Arows in the picture (A Hopfield net with four nodes) have wrong direction. --Bojan PLOJ (talk) 11:37, 6 March 2010 (UTC)[reply]

Initializing the Network

I tried using this article (and some other sources) to create a hopfield net, but I found, in addition to the lack of notes on how to train the network, it is very unclear how the network ought to be initialized. Perhaps psuedocode describing the algorithms involved would be helpful? Chekhov.seagull (talk) 19:29, 8 March 2010 (UTC)[reply]