Talk:Wiener–Khinchin theorem

False autocovariance

The second equation in this article defines the autocovariance of a stationary process, not the autocorrelation. It needs to be normalized by dividing by the autocovariance at t = 0, or just the variance of the process.

No, you are confusing the autocovariance function with the covariace coefficient and the correlation coefficient.
98.81.0.222 (talk) 01:19, 9 July 2012 (UTC)

The literature has different definitions of autocorrelation. Engineers tend to use autocorrelation for the un-normalised variance, whereas statisticians prefer to call that one the autocovariance, and then normalise it to get what *they* (*we*) call the autocorrelation function. 98.109.227.161 (talk) 23:11, 8 November 2012 (UTC)

Autocovariance vs. Autocorrelation

Autocovariance and autocorrelation are not the same thing.
A previous commenter got it wrong, too.
Call the first one C(τ) and call the second R(τ).
Now, the first one is set up so that C(0) = 0, or in electronics engineering terminology, this means that C(τ) does not have a "DC" value, and when we take the Fourier transform of C(τ), we DO NOT get a Dirac delta function in the spectral density S(f) at frequency f = 0.

In contrast, R(τ) CAN have a positive value, or in electronics engineering terminology, it can have a positive "DC" value. (Negative DC values cause other problems.) When we take the Fourier transform of R(τ) in that case, we DO get a Dirac delta function in the spectral density S(f) at frequency f = 0. Hence, S(f) is discontinuous at f = 0, and the derivative of S(f) does not exist at f = 0.

To summarize the relationship between C(τ) and R(τ), we have a simple equation: R(τ) = C(τ) + K, where K is a positive number or possibly zero.
98.81.0.222 (talk) 01:12, 9 July 2012 (UTC)

"Discrepancy"

There is no discrepancy, in the mathematics literature. In slightly more up-to-date mathematics, what that article is trying to say is actually Bochner's theorem, sometimes also called Herglotz's theorem. The autocovariance function is always the Fourier transform of a measure, called the spectral measure of a wide-sense stationary stochastic process. Only when the spectral measure is absolutely continuous, with respect to the Lebesgue measure, does one have a spectral density. For example, if a moving average process has sufficiently long memory, the autocovariance function decay slowly---no spectral density.

Absolute continuity, i.e. the spectral density if it exists, is not that special. The point here is the spectral measure decomposes an stationary process as an stochastic integral of orthogonal increments. Karhunen–Loève theorem does this for continuous time process that are not necessarily stationary.

Does someone know what the original result is? What exactly did Wiener prove? Something probably got distorted during the passage from math to engineering. Mct mht (talk) 09:17, 24 October 2012 (UTC)

I have just been re-reading Wiener's classic *Acta* paper, «Generalized Harmonic Analysis», and his Yellow Peril Book from the War. He studied, before 1930, the sample autocorrelation function of a single realisation, f(t), of a process, and the power spectral density (explicitly explained as a distribution in what would later be the sense of Laurent Schwartz), which he thought of as a generalisation of Schuster's *periodogram*. He called it «The spectrum of a function», or, for a discrete time series, «the spectrum of an array». For him (and the article may be trying to hint at this in an obscure way), whole point was that the autocovariance function, since it is not necessarily square-integrable or integrable, need not have a Fourier transform in the usual sense. To get around this rigorously, he proved that if $f$ is a measurable function such that for all $\tau$, the sample autocorrelation function $\phi(\tau) = \lim_{T \to \infty} \frac 1T\int_0^T f(t+\tau)f(t) dt} exists, then one can use a very tricky kind of principal value integral work-around to prove that what would have been an anti-derivative of the Fourier transform of $\phi$ (if it had existed) can be defined and studied, he called it in 1930 «the integrated spectrum» $S$, it is a lot like a cumulative statistical distribution function, monotonic, value at infinity is the total power of the signal, but not differentiable. From this he (by hand) defined what we now call the power spectral density function, and recognised that it made sense in the sense of distributions as the derivative of $S$. So to be a theorem, the whole point is to see that it is not the Fourier transform in the usual sense, and the usual integral formula diverges.

Thanks for the information. Sounds to me like Wiener's result has been superseded. Bochner's theorem, or the special case of it which applies here, says exactly that. Bochner's theorem is more direct. From what you say, I am guessing Wiener considered the function f(x)/x to take care of the decay-at-infinity issue. In the frequency domain, this becomes the anti-derivative and its distributional derivative is what he called "spectral density". Bochner, on the other hand, approached the problem via unitary representations and came up with a measure that forms a Fourier pair with f(x). Mct mht (talk) 10:19, 6 November 2012 (UTC)

Quite so, except that I don't think Wiener's approach is superseded. They were working at the same time, by the way. Now, this means that the article, in its current state here, is making *false* assertions. The needed hypothesis for the assertions in the article as it stands is that phi is absolutely integrable, which is not always true. Wiener's 1930 formulation (preceded by publications in 1925 and 1926 which however made more restrictive hypotheses) avoids this issue, as I pointed out, and Chatfield, who is careful to be correct in his statements even when he leaves out the proofs, states the Wiener--Khintchine theorem in that fashion. 98.109.227.161 (talk) 23:16, 8 November 2012 (UTC)

The word "independently" as it pertains to Khinchin

IT is not idiomatic English to speak of «independently published». We talk about independent *discoveries*.

So I think I will, later, just remove that one word, «independent».

This will avoid a whole plethora of issues:

Who says it was independent? One would need a reliable source for that. I know of no reliable source that says it was independent.

I have not seen Khinchin's article, so I can only conjecture, but based on reading Wiener's articles and some French articles in 1945 that refer to a publication of Khinchin in 1932, I have an idea of the relationship, and it is not independence. Wiener focusses on the individual sample realisation drawn from a process, which could be a quite arbitrary measurable function. He only discusses the stochastic process under the hypothesis that it is ergodic. That is a stronger assumption than stationary. But it seems that the notion of stationary was only defined in 1932, after Wiener's work, by Khinchin. So it seems likely that Khinchin, in 1934, formulated a version of the Wiener-Khintchine theorem that applied to stationary stochastic processes, not to their sample functions. So this is a shift in focus and a slight generalisation, and the word «independent» is certainly inadequate. 98.109.227.161 (talk) 03:23, 9 November 2012 (UTC)

I googled up a source and added it. Dicklyon (talk) 03:57, 9 November 2012 (UTC)

bah humbug, although some pages are missing, it looks like Nahin makes false mathematical assertions (by ignoring the hypotheses needed to guarantee the convergence of his integrals. As is typical of many textbooks. I downloaded the copy of Khinchin's original paper from Goettingen, and nowhere in that paper is Nahin's version of the theorem written or implied. Khinchin sticks to the formulation with a Stieltjes integral of the spectral decomposition of the autocorrelation function, as it now stands in this wikipedia article, too, using only F, and never passing to the spectral density. Khinchin relies on, and cites, Bochner's theorem, published in 1932, which postdates Wiener's work. Clearly, Khinchin's real contribution was to study this formula for the stochastic processes instead of just their sample functions, and to define the general notion of wide-sense stationary, which the French authors I have seen refer to as «stationary in the sense of Khintchine». 98.109.227.161 (talk) 04:21, 9 November 2012 (UTC)

I'm not saying it's a great source for the math, but it does support the idea of the independent discovery. He could be wrong on that, too, but unless we have a better source that says differently, I'm OK calling it an independent discovery. Dicklyon (talk) 04:25, 9 November 2012 (UTC)

yes, unless an even more reliable source shows up contradicting him, it may as well stand 98.109.227.161 (talk) 04:30, 9 November 2012 (UTC)

here is one link http://www.digizeitschriften.de/dms/img/?PPN=PPN235181684_0109&DMDID=dmdlog4198.109.227.161 (talk) 04:30, 9 November 2012 (UTC)

and here is another link to Khinchin's original paper, showing that his formulation is the one currently installed in this article http://resolver.sub.uni-goettingen.de/purl?GDZPPN00227693398.109.227.161 (talk) 04:30, 9 November 2012 (UTC)

Well, now that I've looked at Khintchine's original article, where he cites Bochner and relies on Bochner's theorem, I see that actually «independent» is really not the appropriate word. Khintchine proved the analogous result in the different context of stochastic processes, whereas Wiener did it for the sample functions. Also, Khintchine was the one who had introduced the concept of stationary stochastic process, so he, with very little pain, extended Wiener'¡s result slightly and proved it in a different context too. That explains why it is not in alphabetical order....98.109.240.7 (talk) 03:40, 14 November 2012 (UTC)

You should publish a history article on that, so we can use it as a source. Dicklyon (talk) 03:42, 14 November 2012 (UTC)

You're right, and until then, one cannot include research or synthesis in a wikipedia article.98.109.240.7 (talk) 16:37, 15 November 2012 (UTC)

I've seen some papers talking about the history of the theorem's development—I was planning on working on the history section of this article, but I've gotten sidetracked since then. I'll see if I can track them down, now that someone else is interested in doing the same thing. Mygskr (talk) 17:02, 15 November 2012 (UTC)

OK, the article I was thinking about is from the old IEEE ASSP Magazine: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=1165596. It doesn't say too much about Wiener or Khinchin, focusing instead on Einstein's contribution. It mentions that Khinchin approached the problem using his theory of stochastic processes, whereas both Einstein and Wiener just look at the sample functions.

Thx for the reference. Now, I have looked at the Einstein article, and he can't be given any credit for 'XXX Theorem" since he is oblivious to the issues that make the proof difficult. Which is fine for physics, but if one is going to talk about a "Theorem", then he is out of the running.198.144.201.135 (talk) 20:28, 15 April 2014 (UTC)

I don't care that much, but I feel I should at least point out that a certain theorem in number theory is still known as Fermat's Last Theorem even though it's almost certain that he did not prove it and did not understand the issues that make the proof difficult. — Preceding unsigned comment added by Mygskr (talk • contribs) 20:41, 15 April 2014 (UTC)

It would not hurt to attribute that one author's opinion that It therefore appears at present that, instead of the “Wiener-Khintchine theorem,” we would be more justified in speaking of the ”Einstein-Wiener-Khintchine theorem.”. But that's as far as we can go if we haven't seen that catch on. Dicklyon (talk) 20:53, 15 April 2014 (UTC)

Einstein versus Wiener-Khinchin, Bochner etc.

Continuing the above discussion... At the moment the article still seems a little dismissive of Einstein's version and focused on the more general "point of Wiener's contribution", perhaps because there is no separate name for Einstein's (and as if Einstein's is a simple corollary of Wiener and Khinchin's, which may be true in a structural sense but this ignores the chronology). Based on the Yaglom article, would something along the following lines be useful to include in the article? "Einstein in 1914 gave a version that is more restrictive in scope than the later 1930s versions of Wiener and Khinchin, but because the significance of Einstein's work was not recognised until later, this version does not have its own name. Therefore the name Wiener-Khinchin theorem is often used for Einstein's version (which is effectively a corollary of the full theorem, albeit one with historical priority) even though Einstein did not address key concerns of Wiener and Khinchin." — Preceding unsigned comment added by 192.76.8.66 (talk) 12:24, 19 October 2023 (UTC)

Bibliography

Not all the in-line citations are to sources which are prestigious and reliable in the sense that you can trust the author has not made severe mathematical mistakes.

I have added a short list of the most prestigious time-series texts which I have used and which are careful not to make false assertions or omit needed hypotheses.

Please don't add your favourite undergrad engineering textbook on signal processing until after you have checked that it is free from false statements. Example: it is just false to say that the autocorrelation function of a wide sense stationary stochastic process has a Fourier transform. Some of them do, others don't. You need a hypothesis..... Fuller and Chatfield are both careful about that, even though they omit the proofs.98.109.227.161 (talk) 05:18, 9 November 2012 (UTC)

Discrepancies in Terminology

It says "Einstein is an example". I think the writer meant an example of someone using the Wiener-Khinchin theorem without worrying about convergence and ya da ya da ya da. I say "I think" because no reference is provided. Since the only work of Einstein's I am aware of that involved stochastic time series type stuff was his 1905 Brownian motion paper and I don't recall anything as mathematically high brow as the Wiener Khinchin thm in there. I personally think the unreferenced Einstein citation should either be removed or a proper citation added. But since I have no idea what the author was referring to I will leave it for someone else to correct. — Preceding unsigned comment added by 69.254.150.101 (talk) 19:52, 1 December 2012 (UTC)

This has since been done (Einstein 1914). — Preceding unsigned comment added by 192.76.8.66 (talk) 11:13, 19 October 2023 (UTC)

What does "putting Sxx almost everywhere" mean?

When I read this:

by taking the derivative of $F$, putting $S_{xx}(f)=F'(f)$ almost everywhere, and the theorem simplifies...

I was like, wuuuhh? Seriously, this can't be standard notation. Can someone reword it so it is actually helpful please? — Preceding unsigned comment added by RatnimSnave (talk • contribs) 17:22, 10 January 2014 (UTC)

It is standard notation since it allows for the same notation to be used with the related cross-spectrum. If x and y are two stochastic processes, you can then define S_xy and S_yx. If x = y, you get S_xx. — Preceding unsigned comment added by 96.38.109.155 (talk) 03:32, 28 February 2014 (UTC)

I linked almost everywhere. I'm not so sure about the "putting" part. Dicklyon (talk) 06:13, 28 February 2014 (UTC)