Cette page appartient aux archives web de l'EPFL et n'est plus tenue à jour.
This page belongs to EPFL's web archive and is no longer updated.

Interacting with Humans I

You are required to read and comment on the following papers.


J. Wright and K. Leyton-Brown, 2010, Beyond equilibria: Predicting human behavior in normal form games, AAAI 2010

J. Wright and K. Leyton-Brown, 2012, Behavorial game-theoretic models: A Bayesian framework for parameter analysis, AAMAS 2012

Posted by Katherine Skirving Larson on Thursday 5 April 2012 at 9:46
These two papers analize various tools of behavioral game theory: they describe four key paradigms of behavioral game theory (QRE, Lk, QLk, CH), evaluate them and propose novel framework for parameter analysis.

As papers point out, human behavior does not follow the concept of the Nash equilibrium. Due to the computational limitation or existence of the multiple equilibria, a human player often adopts strategies that are not game solutions (Nash equilibria). By making models that limit cognitive abilities of agents, behavioral game theory tries to explain and predict human behavior.

The papers investigate the four models (QRE, Lk, QLk, CH) of behavioral game theory and analyze how well they predict human play of normal form games (single stage games). The main contribution of the first paper (Beyond Equilibrium: Predicting Human Behavior in Normal-Form Games) is that it makes comparative analysis of the four models using data sets gathered from various sources; it shows that the best prediction is achieved with QkL technique. However, though the models deeply rely on their parameters, the paper gives no instruction on how to choose them or how sensitive the models are to precision of their parameters. This is addressed in the second paper (Behavioral Game Theoretic Models: A Bayesian Framework For Parameter Analysis), where authors analyze posterior distributions of parameters for two models: QLk and Poisson-CH. The second paper investigates variations of QLk model and shows that one can derive more efficient (more accurate) model with fewer parameters (less complex).
Posted by Goran Radanovic on Sunday 15 April 2012 at 2:27
Throughout the comment, we will refer to [1] as "Beyond..." and [2] as "Behavioral..."

Both of the paper are nicely written and correlated to each other. In particular, the work in [2] is done based on the result obtained in [1]. They addressed an issue that essentially Nash equilibrium, even though it is a solution concept, is not a good way to represent human behavior in (one-shot, unrepeated) normal form game. From this motivation, [1] took four well-known (human behavior) prediction models, i.e. QRE, Level-k, Cognitive hierarchy, QLk and exhaustively analysis them. The analysis was based on the dataset collected by several earlier publications. It turn out that indeed these models generally perform better than Nash equilibrium and QLk generally perform better than the other models. Please note that the notion of "unrepeated" here is important, since if it is a repeated game then there is a high possibility that the players' actions will eventually could reach an equilibrium phase.

However, getting the result from [1] does not mean that the problem is solved. They (the models) basically have parameters that we have to set - which in the end determine the model's performance. For this purpose, [2] suggest a Bayesian approach to analysis and hopefully find parameter settings which will give high prediction performance. Then, basically the results from [1] and [2] can be used by either economist or computer scientist when they define a game, mechanisms design or any kind of agent interaction protocol that involved human player in it.

- Since this is an experimental study, please keep in mind that the result could be different using different dataset, i.e. different games and different human players, especially using players from different background and education.
- In [1], the authors spotted a possible improvement by leaving out Poisson distribution from Cognitive hierarchy, and they did found an improvement using other distribution! This is very clever. But unfortunately they did not explain the rationale behind it.
- In [2], we have found an exhaustive list of model performances. The definition about "efficient frontier" here really helps us on what to look for on the model.

- About the new prediction model that they proposed, QCH in [1] and all models which has prefix "a" in [2], can it be used in practice? More specifically, how can an agent correctly has other agents' precision and level?

Possible improvement/future works:
- About Level-k (and QLk), can we do better if an action for agent level-0 is not random, but possibly one of these (or even a vote of one of these): (i) action which has maximum payoff (ii) action with highest average payoff (iii) action which has highest minimum payoff.
- Since [1] has shown that QCH's performance that originally use Poisson distribution can be improved by the means of other distributions, it would be nice if in [2] they also analyze this.
- Despite all of the works that has been done in [2], the question remains: given that we have no prior observation, how can we have a good prediction model (and their parameter setting) of the human behavior on the game?
Posted by Tri Kurniawan Wijaya on Monday 16 April 2012 at 17:06
First paper takes four behavioral models from literature to answer this question that which would be the best to predict the behavior of humans as agents in normal-form games. These behavioral models include Quantal Response Equilibrium (QRE), level-k, cognitive hierarchy, and quantal level-k. In QRE model, the agents become more likely to make errors as those errors become less costly. So the best response of the agents does not come from a strict maximization and a new form of Nash equilibrium can be introduced in this model. The key idea in level-k model is that agents can perform only a bounded number of iterations of strategic reasoning. After that level agents play randomly. Like level-k, cognitive behavioral model tends to model human brain but it differs from the level-k model in two ways. First, each agent responds perfectly to its beliefs. Second, agents best respond to the full distribution of lower-level types, rather than only to the strategy one level below. Finally, quantal level-k is a combination of QRE and level-k in which agents have a bounded number of iterations of reasoning while they can make errors. The authors claim that the quantal level-k model has better prediction performance than any other model from the literature, over their data.

Second paper, shows how the Bayesian parameter analysis can be used to study the sensitivity of the behavioral game models. Because for all the models introduced and studied in the first paper we need to adjust the parameters. They apply these analyses to investigate the performance of two methods of quantal level-k and Poisson-CH. In contradiction with the results of the first paper, the authors uncover anomalies in quantal level-K.

In fact there are two concerns. First, the second paper lacks the argument, which could show that Bayesian model is the best way to get the parameters. Second, both the model and its parameters are evaluated according to the experimental data. Data from two papers could differ, which could change the performance of the models.
Posted by Laleh Makarem on Tuesday 17 April 2012 at 22:02
The two papers state that a human player might not adopt a Nash equilibrium strategy because of computational limits of herself or her opponents and the uncertainty in finding the Nash equilibrium that the other players will adopt when there are multiple Nash equilibria. In behavioral game theory, based on human cognitive biases and limitations, predictions are made for the actions that a human player will adopt. In the first paper, four key such models are selected and described, namely level-k (Lk), quantal level-k (QLk), cognitive hierarchy (CH), and quantal response equilibrium (QRE).
Among these models, Lk and CH capture the idea of bounded iterated reasoning of agents, QRE captures the idea of cost-proportional errors and QLk combined these two properties.
The main question that this paper tries to answer is: which of these for models is best for human behavior model prediction in normal-form games?
The paper conducts an experimental analysis for comparing these four models and also for answering other important questions. The study shows that QLk which capture both cost-proportional errors and bounded iterated reasoning properties outperforms other models. It also shows that behavioral models predict better the human's actions in normal-from games than NE. Other observations are: 1) the assumption that the agent levels have a Poisson distribution is not helpful in the CH model; 2) restricting the level of agents to at most 2 can reduce the prediction accuracy; 3) the "money illusion" effect holds for each individual dataset but, for the combined one payoff normalization improves the prediction power of QRE; and 4) Heterogeneity does not improve predictions. Finally, the paper suggests that QLk is the best model that can be used and quantal cognitive hierarchy should be used when the analysis of the parameters and the ability to vary the parameter numbers are important.
Since all the experimental data are for two player games, it is interesting to see how the results can be generalized to multiple player games.

In the second paper the authors propose a baysian framework for estimating and analyzing the posterior distribution of parameters of behavioral models based on prior beliefs and experimental data. Among the previously studies models, two models are studied here, namely Poisson-CH (for comparison with the literature) and QLk (the best performing model based on the first paper). For Poisson-CH model, it is observed that the recommended parameter value in the literature is not correct.
For QLk several surprising results are observed that contradict with the expectations. Then several model variations are considered to investigate the properties of QLk. For example, the study shows that the flexibility in describing the level distribution is more important than the flexibility achieved by inhomogeneous precisions and general precision beliefs. Finally, the more restricted model of ah-QCH3 (QCH model with accurate precision belief, homogeneous precision, and agent levels between 0 and 3) is shown to reasonably perform better than the other models w.r.t the number of parameters.
Posted by Mehdi Riahi on Wednesday 18 April 2012 at 0:12