Talk:Analytic hierarchy process

Strong POV comes through in text

I'm new here and I have to say that this article reads like it was written by one of Saaty's grad students. It's full of value-laden adjectives (positive for AHP, negative for everything else), and presents no account (balanced or otherwise) of the many criticisms of AHP. Given that rank reversal is considered a prima facie refutation of AHP by many economists, mathematicians, and operations researchers, any unbiased encycolpedia article should address it. DMTate (talk) 21:07, 7 July 2008 (UTC)

So others can better understand your concerns, please be more specific about the "value-laden adjectives (positive for AHP, negative for everything else)" that you see here. Lou Sander (talk) 17:00, 8 July 2008 (UTC)

DMTate and I must not be looking at the same article. I don't see any mention at all of "anything else," let alone any value-laden negative adjectives about it. There is a large criticisms section at the end of the article. Unfortunately it doesn't cite anything that can be accessed online. 74.1.175.146 (talk) 23:51, 9 July 2008 (UTC)

The Criticisms section references eight articles from the 1990s that contain a long debate about AHP. The section includes a single quotation from that debate (J.S. Dyer), with no other context, summary of the debate, etc. I propose to delete the quote, but retain the reference to the article from which it was taken. Skyrocket654 (talk) 15:13, 12 July 2008 (UTC)

I agree with Skyrocket654 that this quotation seems out of place in this part of the Criticisms section that seems to be more about laying the groundwork with a list of reference articles. Probably it should be removed, but the reference retained. MathDame (talk) 19:25, 22 July 2008 (UTC)

Done. I also propose to remove the Holder quote that follows the above, for the same reasons. It is unreferenced, but presumably is contained in one of the two citations that mention Holder. Skyrocket654 (talk) 16:41, 23 July 2008 (UTC)

BTW, I completely agree with what DMTate wrote above. It reads like a brochure for AHP; it always puts a positive spin on things.

E.g., "It is the nature of the AHP to promote focused discussions about difficult aspects of the decisions to which it is applied. Such discussions encourage the communication of differences, which in turn encourages cooperation, compromise, and agreement among the members of the group."

Even the tone of the first sentence of the criticism section biases the section. It suggests that there is broad consensus that AHP is sound, but somehow there are still criticisms. Burkander (talk) 20:51, 1 October 2008 (UTC)

1) DMTate talked about "value laden adjectives." Burkander "completely agrees" with him, and presents an illustration. The only adjectives in the illustration are "focused" and "difficult;" IMHO those words aren't very value-laden.

2) There IS broad consensus that AHP is sound. The reference more than "suggests" it, it states it outright. Further evidence of soundness is in the widespread use demonstrated in the Uses and Applications Section.

3) There are definitely criticisms of the method, mostly advanced in academic journals from many years ago. About a dozen of them are referenced in the Criticisms section. They don't lend themselves very well to exposition in an encyclopedia article, and neither do the answers or responses to them. Reference #21 deals with the whole matter at length; maybe it should be available online.

The reference is now available online, and is linked to in the article. Lou Sander (talk) 13:38, 16 October 2008 (UTC)

People who feel that the method is unsound are free not to use it, and some of them don't. But a lot of people DO use it, and they aren't unaware of the criticisms. Lou Sander (talk) 02:56, 2 October 2008 (UTC)

Implementation / External links

I added a very short section mentioning the use of the AHP software and showing one of the gadgets. It references both of the companies that previously had entries in External links. I removed those links, which were both a bit promotional in their descriptions. I also removed an External link that duplicated one of the references in the body of the article. Good Cop (talk) 22:14, 27 July 2008 (UTC)

Rank reversal, etc.

In response to calls for more specific information, I added a large subsection about rank reversal. I also did some minor rejiggering of headings. Lou Sander (talk) 19:43, 12 October 2008 (UTC)

Somehow, a key article in Operations Research that I wrote with Saul Gass was not referenced. It explains rank reversal and the ideal mode in great detail. I've also added an example (to this article) of Presidential elections where rank reversals have been present in the recent past.

The reference to Arrow's Impossibility Theorem was very vague. In fact, Arrow's Impossibility Theorem is NOT impossible if ratio measure, such as that used in AHP is applied.

Schenkerman's article is based on a wrong assumption by Schenkerman. AHP will produce the exact results Shenkerman was looking for if applied correctly.

(Ernest Forman 23 November 2008). —Preceding unsigned comment added by EForman (talk • contribs) 14:51, 23 November 2008 (UTC)

The following material was inappropriately put in the article itself. I moved it here. Lou Sander (talk) 01:08, 1 December 2008 (UTC)

Comment: The above paragraph does not make sense to me. Your original table simply did not represent the number of available dresses of each type. As your example continues, the customer now takes into account the number of available dresses of each type (e.g. relative scarcity of each alternative), and so clearly the decision table should be modified to include this new criterion. With this new criterion in place, priorities can be re-assigned and alternatives (there will be no additional ones) re-evaluated. If now the ranking of alternatives is different, e.g. reverses, this is due to the additional criterion (relative scarcity). In fact, referring to the discovery of new dresses (of existing types) as "additional alternatives" or "copies" makes very little sense to me: a criterion was added, not an alternative. With respect to the subsequent example (the 2004 elections), there are no "copies" involved, rather you discuss the effect of a dominated alternative on the relative rankings of the dominating alternatives. 69.217.120.82 (talk)

Neutrality & Tone

The tone of this article reads a bit like a press release rather than a neutral article. In particular the section on criticisms is written in a manner that causes one to discount the criticisms. This section could also benefit from some mention of Arrow's Impossiblity Theorem192.237.29.125 (talk) 17:46, 18 November 2008 (UTC)

Strongly disagree. The criticisms are given their just due. Remove the neutrality tag, please. Good Cop (talk) 21:43, 18 November 2008 (UTC) —Preceding unsigned comment added by DCLawyer (talk • contribs)

The "unsigned" notation above refers to my moving this section to the end of the page. It formerly was out of place at the beginning. Sorry if anybody got confused. DCLawyer (talk) 14:09, 19 November 2008 (UTC)

Disagree. Remove the Neutrality tag. The anon calls the article "promotional" but provides no examples. The criticisms involving rank reversal are the major criticisms of the method. They are thoroughly discussed, with references. The "other criticisms" are old and/or have no discernable following. If there is any published material relating Arrow's theorem to the Analytical Hierarchy Process, it isn't cited. DCLawyer (talk) 14:09, 19 November 2008 (UTC)

This sentence for starters as the lead in to the section pushes people to doubt the criticisms:

Although the Analytic Hierarchy Process has been the subject of many research papers and the general consensus is that the technique is both technically valid and practically useful, there are critics of the method[4]

The sentence is from a cited source that is available online. It seems to sum up the state of affairs regarding criticism of AHP: the process is extremely widely used; its inventor keeps receiving important awards for inventing it; AND there are still occasional papers that advance one criticism or another. IMHO, those occasional papers are given undue weight in this section of the article. Skyrocket654 (talk) 00:01, 22 November 2008 (UTC)

In the early 1990s a series of debates between critics and proponents of AHP was published in Management Science.

This is so, and quite a number of them are cited as references. Given the age of this debate and the subsequent worldwide adoption of AHP by legions of serious users, there may be an undue weight problem here, too. But nobody is complaining about it. Skyrocket654 (talk) 00:01, 22 November 2008 (UTC)

This sentence as the lead in to the rank reversal section raises doubts about the criticism before raising explaining what it is:

"Most criticisms of AHP have involved a phenomenon called "rank reversal." Many people hear about rank reversal and assume that there is some sort of proven principle about it that needs to be upheld in making decisions. That unwarranted assumption has led to much misunderstanding of AHP and other decision making techniques."

There are plenty of doubts about the criticism, and they have been raised in numerous places. IMHO the subsequent long explanation of rank reversal, coupled with the online availability of a very long treatise on the subject, pretty much lay out the situation. Skyrocket654 (talk) 00:01, 22 November 2008 (UTC)

As for the Arrow's Impossiblity Theorem, most of the criticisms aren't easily availible publicly - the journarls are paid subscriptions. But the theorem itself as currently defined on wikipedia is a criticism:

"[Arrow's Impossiblity Theorem] demonstrates that no voting system can convert the ranked preferences of individuals into a community-wide ranking while also meeting a certain set of reasonable criteria with three or more discrete options to choose from" —Preceding unsigned comment added by 192.237.29.125 (talk) 17:26, 21 November 2008 (UTC)

If an editor wants to say that somebody's theorem involves criticism of AHP, it is their responsibility to find citations of that criticism in verifiable published sources. We can't just, on our own, cite somebody's theory and conclude that it must apply to AHP or anything else. Skyrocket654 (talk) 00:01, 22 November 2008 (UTC)

It is a proven fact, verifiable in any kitchen, that metal objects sink in water. This is NOT a legitimate criticism of metal ships. DCLawyer (talk) 06:07, 22 November 2008 (UTC)

Remove the neutrality tag. Except for rank reversal, the other criticisms are pretty obviously given undue weight. It would be good if all editors of the criticisms section would read WP:Undue and temper their enthusiasm accordingly. Skyrocket654 (talk) 14:23, 29 November 2008 (UTC)

Remove the neutrality tag. I agree that the other criticisms are given undue weight. MathDame (talk) 20:45, 2 December 2008 (UTC)

The biggest issue here is tone of the entire article. This reads like a press release not an encyclopedia entry.tone

Many editors seem to disagree with you. Please be specific about what you mean. Maybe use this page to quote something verbatim from the article and show us how you think it "reads like a press release."

With regards to the Criticisms section, it is pretty clear that criticisms from very old journal articles that failed to gain a following are described in inappropriate depth. The detailed quotation from "AHP critic Stan Schenkerman" is an example of this. His viewpoint is not significant or prominent. It is a ten-year-old view of one person, now deceased, yet it is described as "another problem {with AHP]." Not only that, but it concludes that "the decision maker relying on AHP or these variants can be seriously misled." WP:UNDUE says that such tiny minority views shouldn't even be in the article. IMHO, Schenkerman's point should perhaps be mentioned and cited, but that's about all. Skyrocket654 (talk) 16:57, 11 December 2008 (UTC)

I removed the tag, per the above discussion. DCLawyer (talk) 16:21, 13 December 2008 (UTC)

"SUBSTANTIATE"

I removed a bunch of these from the article. They don't belong there. There ARE ways to question material, but this isn't one of them. Skyrocket654 (talk) 17:01, 11 December 2008 (UTC)

Criticisms section

In keeping with the discussion in several sections above, I recast the criticisms section to fix the undue weight being given to minority viewpoints. The unsupported claim about Arrow's Impossibility Theorem was removed because it seems to be original research. The material in the 1997 and 2006 articles was reduced to a single (hopefully fair) sentence on each, and the citations were retained. The INFORMS Impact Prize was put into a footnote. Everything but Rank Reversal was collected and put at the beginning of the section. DCLawyer (talk) 16:02, 13 December 2008 (UTC)

I also removed the "unwarranted" from the "unwarranted assumption" about rank reversal. The word doesn't appear in the cited reference. DCLawyer (talk) 16:26, 13 December 2008 (UTC)

The article briefly mentioned a paper by Forman and Gass. I found a copy of it, and it has a lot of good stuff about the academic criticisms. I mentioned it more prominently in the Criticisms section, including a link to an online copy HERE. Lou Sander (talk) 04:11, 16 December 2008 (UTC)

I made a change to the first paragraph of this section. Someone had expressed concern that the previous lead sentence was based on a practical magazine for users, rather than on a peer-reviewed paper. It is now based on a peer-reviewed paper, with additional comment from the practical magazine.

IMHO the criticisms section is now thoroughly referenced to strong sources, and describes the situation completely and fairly: AHP's undeniably widespread use and thorough vetting are first briefly mentioned, with solid references. Then twelve specific critcal papers are mentioned and referenced, with descriptions of the main points of the most recent ones. (Though a case can be made that as tiny minority views the recent ones are given undue weight, nobody is complaining about it.) Then there is a long subsection explaining Rank Reversal, the phenomenon underlying almost all the criticisms. Finally, two full-text references that summarize and analyze the criticisms are provided -- one from a peer-reviewed journal (Forman and Gass), and one from a scholarly foundation text on AHP. Lou Sander (talk) 15:27, 19 December 2008 (UTC)

I agree that this is getting to be a nice piece of work. Good job, Lou Sander. MathDame (talk) 04:40, 20 December 2008 (UTC)

Support for Theory

An anonymous IP editor has split this off from the Criticisms section. There were no edit summaries. The user has a slightly questionable history. The material is about criticisms of AHP, and there doesn't seem to be any reason to split it off. I propose to put it back in the criticisms section. MathDame (talk) 17:05, 4 January 2009 (UTC)

I agree with MathDame. There's no sense in breaking off the new section. A week has passed with no comment. Put it as it was. Lou Sander (talk) 17:17, 11 January 2009 (UTC)

Support. Combine the two. 74.1.175.146 (talk) 01:29, 15 January 2009 (UTC)

Done.

MathDame (talk) 15:22, 15 January 2009 (UTC)

Summary section

I've redone the first part of the summary section without (I hope) changing its essence. The steps now correspond to those in the cited reference. I removed two claims that were challenged over ten months ago and well over a year ago and never responded to. Lou Sander (talk) 18:24, 11 January 2009 (UTC)