Good judgment comes from experience, and experience… well that comes from poor judgment

Bernard Baruch

---

„Pairwise comparison generally refers to any process of comparing entities in pairs to judge which of each entity is preferred, or has a greater amount of some quantitative property” (Wikipedia). In my opinion, the Pairwise Comparisons Method is an essential part of Decision Theory. Moreover, it is a part of the theoretical basis of decision support systems. Although the pairwise comparisons method (PCM) has a very long story, there is still some room for improvement.

Understanding The Analytic Hierarchy Process 

Foreword

In more than fifty years, the Analytical Hierarchy Process (AHP) has gone well beyond its initial formulation and has become a body of knowledge on the hierarchical arrangement of criteria and alternatives, the use of pairwise com parisons, and the aggregation of priorities according to the hierarchy. Possibly due to the generality of the approach, the AHP framework has been a fertile ground for researchers. Just to give an idea of this wealth, one can easily count more than twenty methods to derive a weight vector from a pairwise compari son matrix, and again more than twenty indices to estimate the inconsistency of pairwise comparisons. When Konrad, years ago, disclosed his intention to write a book on the AHP, I began to wait. I had known him for some time, so I knew that it was a matter of “when” and not a matter of “if”. Of course, the more time was passing, the greater my expectations were growing. Now, after much waiting, I can say that they are completely fulfilled. This book distills and presents the most relevant results in a unique manner since it is the book which more closely offers a formal perspective on the method, and does not avoid digressions into fields like measurement theory and graph theory. Albeit neglected in most presentations of the AHP, it is undeniable that this supplementary material helps the reader understand (as the title of the book correctly suggests) the method and not feel it slip away, like sand through the hands. I maintain that this book has two additional merits: its presentation of the AHP is wide and inclusive. It is wide because it succeeds in presenting the AHP as an organic set of methods. If, on one hand, it is certainly impossible to present all the contri butions related to the AHP, on the other hand, it is necessary to make the reader aware of their existence (and whet the appetite) by means of carefully selected references to the relevant literature. This book is inclusive in the sense that it presents concepts whose relevance goes beyond the AHP. For example, it is nowadays recognized that the concept of pairwise comparison, here defined in the form of a pairwise comparison matrix, is pervasive in the entire field of decision analysis, and not only there. It is auspicable and reasonable to expect that this book will also serve as a bridge between different communities. The book has two types of uses. It can be read from front matter to back matter by those with no previous knowledge of the AHP and can be seen as a “cookbook”, to always keep on your shelf. Each chapter is structured in such a way that makes it a self-standing story which, with some caution on the notation, is also self-contained. In the final analysis, I believe that, whatever type of reader you are, this book will help you understand the AHP.

Matteo Brunelli
Trento, Italy
15 March 2020

Resilient heuristic aggregation of judgments in the pairwise comparisons method

Abstract

In decision-making methods, it is common to assume that the experts are honest and professional. However, this is not the case when one or more experts in the pairwise-based group decision-making framework, such as the group analytic hierarchy process, try to manipulate results in their favor. This paper aims to introduce two heuristics enabling detection of manipulators and minimizing their effect on the group consensus by diminishing their weights. The first heuristic is based on the assumption that manipulators will provide judgments that can be considered outliers with respect to those of the other experts in the group. The second heuristic assumes that dishonest judgments are less consistent than the average consistency of the group. Both approaches are illustrated with numerical examples and simulations.

Almost optimal manipulation of pairwise comparisons of alternatives

Abstract

The role of an expert in the decision-making process is crucial. If we ask an expert to help us to make a decision we assume their honesty. But what if the expert is dishonest? Then, the answer on how difficult it is for an expert to provide manipulated data in a given case of decision-making process becomes essential. In the presented work, we consider manipulation of a ranking obtained by the Geometric Mean Method applied to a pairwise comparisons matrix. More specifically, we propose an algorithm for finding an almost optimal way to swap the positions of two selected alternatives in a ranking. We also define a new index which measures how difficult such manipulation is in a given case.

On the derivation of weights from incomplete pairwise comparisons matrices via spanning trees with crisp and fuzzy confidence levels

Abstract

In this paper, we propose a new method for the derivation of a priority vector from an incomplete pairwise comparisons (PC) matrix. We assume that each entry of a PC matrix provided by an expert is also evaluated in terms of the expert's confidence in a particular judgment. Then, from corresponding graph representations of a given PC matrix, all spanning trees are found. For each spanning tree, a unique priority vector is obtained with the weight corresponding to the confidence levels of entries that constitute this tree. At the end, the final priority vector is obtained through an aggregation of priority vectors achieved from all spanning trees. Confidence levels are modeled by real (crisp) numbers and triangular fuzzy numbers. Numerical examples and comparisons with other methods are also provided. Last, but not least, we introduce a new formula for an upper bound of the number of spanning trees, so that a decision maker gains knowledge (in advance) on how computationally demanding the proposed method is for a given PC matrix.

On the similarity between ranking vectors in the pairwise comparison method

Abstract

There are many priority deriving methods for pairwise comparison (PC) matrices. It is known that when these matrices are consistent all these methods result in the same priority vector. However, when they are inconsistent, the results may vary. The presented work formulates an estimation of the difference between priority vectors in the two most popular ranking methods: the eigenvalue method and the geometric mean method. The estimation provided refers to the inconsistency of the PC matrix. Theoretical considerations are accompanied by Monte Carlo experiments showing the discrepancy between the values of both methods.

Inconsistency indices for incomplete pairwise comparisons matrices

Abstract

Comparing alternatives in pairs is a very well known technique of ranking creation. The answer to how reliable and trustworthy ranking is depends on the inconsistency of the data from which it was created. There are many indices used for determining the level of inconsistency among compared alternatives. Unfortunately, most of them assume that the set of comparisons is complete, i.e. every single alternative is compared to each other. This is not true and the ranking must sometimes be made based on incomplete data. In order to fill this gap, this work aims to adapt the selected twelve existing inconsistency indices for the purpose of analyzing incomplete data sets. The modified indices are subjected to Monte Carlo experiments. Those of them that achieved the best results in the experiments carried out are recommended for use in practice.

Inconsistency in the ordinal pairwise comparisons method with and without ties

Abstract

Comparing alternatives in pairs is a well-known method used to create ranking. Experts are asked to perform a series of binary comparisons and then, using mathematical methods, the final ranking is constructed. Experts conduct a series of single assessments, however, they may not always be consistent. The level of inconsistency among individual assessments is widely accepted as a measure of the ranking quality. The higher the ranking quality, the higher its credibility. One of the earliest and most widespread inconsistency indices is the consistency coefficient defined by Kendall and Babington Smith. In their work, the authors consider binary pairwise comparisons, i.e., those where the result of an individual comparison can only be better or worse. In the presented work, the maximal number of inconsistent triads in the set of ordinal pairwise comparisons with ties of arbitrary size is determined (formula 14). This, in turn, opens the possibility of effectively extending the Kendall and Babington Smith index to pairwise comparisons, where the result of an individual comparison can be: better, worse or equal. Hence, this effectively extends the use of this index to the Analytic Hierarchy Process and other quantitative methods based on comparing alternatives in pairs. The work also introduces the notions of a generalized tournament and a double tournament as graphs that model ordinal pairwise comparisons with ties and the maximally inconsistent set of pairwise comparisons with ties, respectively. The relationship between the most inconsistent set of pairwise comparisons with ties and the set cover problem is also shown.

Notes on order preservation and consistency in AHP

Abstract

The pairwise comparisons method is a convenient tool used when the relative order among different concepts (alternatives) needs to be determined. One popular implementation of the method is based on solving an eigenvalue problem for the pairwise comparisons matrix. In such cases the ranking result for the principal eigenvector of the pairwise comparisons matrix is adopted, while the eigenvalue is used to determine the index of inconsistency. A lot of research has been devoted to the critical analysis of the eigenvalue based approach. One of them is the work of Bana e Costa and Vansnick (2008). In their work, the authors define the conditions of order preservation (COP) and show that even for sufficiently consistent pairwise comparisons matrices, this condition cannot be met. The presented work defines more precise criteria for determining when the COP is met. To formulate the criteria, an error factor is used describing how far the input to the ranking procedure is from the ranking result. The relationship between the Saaty consistency index and COP is also discussed.

On the Properties of the Priority Deriving Procedure in the Pairwise Comparisons Method

Abstract

The pairwise comparisons method can be used when the relative order of preferences among different concepts (alternatives) needs to be determined. There are several popular imple- mentations of this method, including the Eigenvector Method, the Least Squares Method, the Chi Squares Method and others. Each of the above methods comes with one or more inconsistency in- dices that help to decide whether the consistency of input guarantees obtaining a reliable output, thus taking the optimal decision. This article explores the relationship between inconsistency of input and error of output. An error describes to what extent the obtained results correspond to the single expert’s assessments. On the basis of the inconsistency and the error, two properties of the weight deriving procedure are formulated. These properties are proven for eigenvector method and Koczkodaj’s inconsistency index. Several estimates using Koczkodaj’s inconsistency index for a principal eigenvalue, Saaty’s inconsistency index and the Condition of Order Preservation are also provided.

Notes on the existence of a solution in the pairwise comparisons method using the heuristic rating estimation approach

Abstract

Pairwise comparisons (PC) is a well-known method for modeling the subjective preferences of a decision maker. The method is very often used in the models of voting systems, social choice theory, decision techniques (such as AHP - Analytic Hierarchy Process) or multi-agent AI systems. In this approach, a set of paired comparisons is transformed into one overall ranking of alternatives. Very often, only the results of individual comparisons are given, whilst the weights (indicators of significance) of the alternatives need to be computed. According to Heuristic Rating Estimation (HRE), the new approach discussed in the article, besides the results of comparisons, the weights of some alternatives can also be a priori known. Although HRE uses a similar method to the popular AHP technique to compute the weights of individual alternatives, the solution obtained is not always positive and real. This article tries to answer the question of when such a correct solution exists. Hence, the sufficient condition for the existence of a positive and real solution in the HRE approach is formulated and proven. The influence of inconsistency in the paired comparisons set for the existence of a solution is also discussed.

The New Triad based Inconsistency Indices for Pairwise Comparisons

Abstract

Pairwise comparisons are widely recognized method supporting decision making process based on the subjective judgments. The key to this method is the notion of inconsistency that has a significant impact on the reliability of results. Inconsistency is expressed by means of inconsistency indices. Depending on their construction, such indices may pay attention to different aspects of the set of pairwise comparisons. The family of indices proposed in this article tries to combine the advantages coming from different indices, thereby increases the expressiveness of the family elements. The newly introduced notion of equivalence can help in comparing the indices and identifying their common properties

Tender with Success – The Pairwise Comparisons Approach

Abstract

Organization of a tender is not easy. Preparation of the relevant specification, taking into account the non-price criteria, implementation of the objective and fair assessment procedure, and last but not least, selecting a satisfactory offer are in practice a considerable challenge. In meeting this challenge appropriate multi-criteria assessment models can help. Models that can cope with different kinds of tangible and intangible criteria. The paper presents the hierarchical bid assessment (HBA) model of making decision in a tender procedure based on the pairwise comparisons method. It combines structural elements known from AHP with the Heuristic Rating Estimation approach. Two different schemes of rating tangible and intangible attributes are proposed. The notion of the success of the customer is defined and the practical method for its use is proposed. Theoretical considerations are illustrated in the relevant example.

Heuristic rating estimation – geometric approach

Abstract

Heuristic Rating Estimation (HRE) is a newly proposed method that supports decisions analysis based on the use of pairwise comparisons. It allows the ranking values of some alternatives (herein referred to as concepts) to be initially known, whilst ranks for other concepts have yet to be estimated. To calculate the missing ranks it is assumed that the priority of every single concept can be determined as the weighted arithmetic mean of the priorities of all the other concepts. It has been shown that the problem has an admissible solution if the inconsistency of the pairwise comparisons is not too high. The proposed approach adopts heuristics according to which a weighted geometric mean is used to determine the missing priorities. In this approach, despite increased complexity, a solution always exists and its existence does not depend on the inconsistency of the input matrix. Thus, the presented approach might be appropriate for a larger number of problems than previous methods. Moreover, it turns out that the geometric approach, as proposed in the article, can be optimal. The optimality condition is presented in the form of a corresponding theorem. A formal definition of the proposed geometric heuristics is accompanied by two numerical examples.

On the Quality Evaluation of Scientific Entities in Poland supported by Consistency-Driven Pairwise Comparisons Method

Abstract

Comparison, rating, and ranking of alternative solutions, in case of multicriteria evaluations, have been an eternal focus of operations research and optimization theory. Numerous approaches at practical solving the multicriteria ranking problem. The recent focus of interest in this domain was the event of parametric evaluation of research entities in Poland. The principal methodology was based on pairwise comparisons. For each single comparison, four criteria have been used. One of the controversial points of the assumed approach was that the weights of these criteria were arbitrary. The main focus of this study is to put forward a theoretically justified way of extracting weights from the opinions of domain experts. Theoretical bases for the whole procedure are based on a survey and its experimental results. Discussion and comparison of the two resulting sets of weights and the computed inconsistency indicator are discussed.

Heuristic Rating Estimation approach to the pairwise comparisons method

Abstract

The Heuristic Ratio Estimation (HRE) approach proposes a new way of using the pairwise comparisons matrix. It allows the assump- tion that some alternatives (herein referred to as concepts) are known and fixed, hence the weight vector needs to be estimated only for the other unknown values. The main purpose of this paper is to extend the previously proposed iterative HRE algorithm and present all the heuris- tics that create a generalized approach. Theoretical considerations are accompanied by a few numerical examples demonstrating how the selected heuristics can be used in practice.