Talk:Hopfield network

Further Correction - symmetric weights

My error - I misread the context of his statement. The condition of symmetric weights guarantees that following the update rule makes energy a monotonically decreasing function, which guarantees convergence to local minima, however, non-symmetric weights do not seem to impare the use of the network as a content-addressable memory system.

Correction - connections need not be symmetric!

If you refer to the origina Hopfield paper ( citied at the bottom of the page ) he discusses the performance of networks with the "special case" of symmetric weights, but says that the network performs just as well with non-symmetric weights. Specifically he says: "The flow in phase space produced by this model algorithm has the properties necessary for a content-addressable memory whether or not Tij is symmetric" (Hopfield, 1982, p. 2556)

That's right. He says in the 1982 paper (and repeats in the ones from 1984 and 1986) that the weights should be more or less symmetric in order to converge. --Ben ^T/_C 14:27, 5 July 2007 (UTC)

Connection between Hopfield Net and Ising model?

Hello!

I had some classes this week which involved the definitions of Hopfield networks and Ising model, and came here to look for further information/links.

There is a link in this article to Ising model, but nothing is written in the article body that explains the connections between the two concepts, maybe someone could fill that gap in?

(I'll try after I've studied enough to understand the connection myself).

Cheers

The Ising model is a model of ferromagnetism. Atoms are bipolar (i.e. either positive or negative) and they have connections and local interactions of atoms can lead to some state transitions on a global level. They are the theoretical foundation of Hopfield Networks and Hopfield specifically mentions them and changes the atoms to McCulloch-Pitts neurons, i.e. he gives them a threshold. --Ben ^T/_C 14:33, 5 July 2007 (UTC)

Definitions?

The relation between the a[i]'s and the s[i]'s is not clear. Are the a[i]'s just the updated values of the s[i]'s? In that case, why not call them both s[i]?

Another terminological matter: The article says

Hopfield nets can either have units that take on values of 1 or -1, or units that take on values of 1 or 0.

and goes on to give the updating rules in the two cases. This seems like to much attention to a trivial matter of scaling. I would suggest choosing one convention or the other for the article and then mentioning that the other convention is also used. --Macrakis 16:21, 15 August 2006 (UTC)

I also don't find it that important, whether they are ${0,1}^{n}$ or ${-1,1}^{n}$. But I find it important that units can be also continuous. Bipolar units are only one particular case studied in Hopfield's papers. --Ben ^T/_C 14:37, 5 July 2007 (UTC)

Energy formula

Currently energy is written as:

$E=-{\frac {1}{2}}\sum _{i<j}{w_{ij}{s_{i}}{s_{j}}}+\sum _{i}{\theta _{i}\ s_{i}}$

I feel this is incorrect. Either removing 1/2

$E=-\sum _{i<j}{w_{ij}{s_{i}}{s_{j}}}+\sum _{i}{\theta _{i}\ s_{i}}$

or summing over all $i$ and $j$

$E=-{\frac {1}{2}}\sum _{i,j}{w_{ij}{s_{i}}{s_{j}}}+\sum _{i}{\theta _{i}\ s_{i}}$

would fix the problem. But I'm not so confident to modify the main text. I'd appreciate if somebody could check it. -- i agree, and i've changed it, (before looking here) I T.A a neural networks course... you can easily see this be derivating E w.r.t S_j to get h_j

right. --Ben ^T/_C 14:38, 5 July 2007 (UTC)

energy

I have an argument on energy function. Sometimes threshold is a more complicated function and we cannot easily incorporate it into Energy function. I mean as I have seen in "Associative memories - the Hopfield net", it should not contain this term:

$\sum _{i}{\theta _{i}\ s_{i}}$

Am I right? ±±±±±±±