(C) PLOS One
This story was originally published by PLOS One and is unaltered.
. . . . . . . . . .



Interaction between games give rise to the evolution of moral norms of cooperation [1]

['Mohammad Salahshour', 'Max Planck Institute For Mathematics In The Sciences', 'Leipzig', 'Max Planck Institute Of Animal Behavior', 'Radolfzell', 'Centre For The Advanced Study Of Collective Behaviour', 'University Of Konstanz', 'Konstanz', 'Department Of Biology']

Date: 2022-10

Abstract In many biological populations, such as human groups, individuals face a complex strategic setting, where they need to make strategic decisions over a diverse set of issues and their behavior in one strategic context can affect their decisions in another. This raises the question of how the interaction between different strategic contexts affects individuals’ strategic choices and social norms? To address this question, I introduce a framework where individuals play two games with different structures and decide upon their strategy in a second game based on their knowledge of their opponent’s strategy in the first game. I consider both multistage games, where the same opponents play the two games consecutively, and reputation-based model, where individuals play their two games with different opponents but receive information about their opponent’s strategy. By considering a case where the first game is a social dilemma, I show that when the second game is a coordination or anti-coordination game, the Nash equilibrium of the coupled game can be decomposed into two classes, a defective equilibrium which is composed of two simple equilibrium of the two games, and a cooperative equilibrium, in which coupling between the two games emerge and sustain cooperation in the social dilemma. For the existence of the cooperative equilibrium, the cost of cooperation should be smaller than a value determined by the structure of the second game. Investigation of the evolutionary dynamics shows that a cooperative fixed point exists when the second game belongs to coordination or anti-coordination class in a mixed population. However, the basin of attraction of the cooperative fixed point is much smaller for the coordination class, and this fixed point disappears in a structured population. When the second game belongs to the anti-coordination class, the system possesses a spontaneous symmetry-breaking phase transition above which the symmetry between cooperation and defection breaks. A set of cooperation supporting moral norms emerges according to which cooperation stands out as a valuable trait. Notably, the moral system also brings a more efficient allocation of resources in the second game. This observation suggests a moral system has two different roles: Promotion of cooperation, which is against individuals’ self-interest but beneficial for the population, and promotion of organization and order, which is at both the population’s and the individual’s self-interest. Interestingly, the latter acts like a Trojan horse: Once established out of individuals’ self-interest, it brings the former with itself. Importantly, the fact that the evolution of moral norms depends only on the cost of cooperation and is independent of the benefit of cooperation implies that moral norms can be harmful and incur a pure collective cost, yet they are just as effective in promoting order and organization. Finally, the model predicts that recognition noise can have a surprisingly positive effect on the evolution of moral norms and facilitates cooperation in the Snow Drift game in structured populations.

Author summary How do moral norms spontaneously evolve in the presence of selfish incentives? An answer to this question is provided by the observation that moral systems have two distinct functions: Besides encouraging self-sacrificing cooperation, they also bring organization and order into the societies. In contrast to the former, which is costly for the individuals but beneficial for the group, the latter is beneficial for both the group and the individuals. A simple evolutionary model suggests this latter aspect is what makes a moral system evolve based on the individuals’ self-interest. However, a moral system behaves like a Trojan horse: Once established out of the individuals’ self-interest to promote order and organization, it also brings self-sacrificing cooperation.

Citation: Salahshour M (2022) Interaction between games give rise to the evolution of moral norms of cooperation. PLoS Comput Biol 18(9): e1010429. https://doi.org/10.1371/journal.pcbi.1010429 Editor: Fernando Santos, University of Amsterdam, NETHERLANDS Received: April 6, 2021; Accepted: July 21, 2022; Published: September 29, 2022 Copyright: © 2022 Mohammad Salahshour. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability: All relevant data are within the manuscript and its Supporting information files. Funding: The author acknowledges funding from Alexander von Humboldt Foundation in the framework of the Sofja Kovalevskaja Award endowed by the German Federal Ministry of Education and Research and the Deutsche Forschungsgemeinschaft (German Research Foundation) under Germany’s Excellence Strategy EXC 2117-422037984 for funding during parts of this research. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing interests: The authors have declared that no competing interests exist.

Introduction Although beneficial for the group, cooperation is costly for the individuals, and thus, constitutes a social dilemma: Following their self-interest, individuals should refrain from cooperation. This leaves everybody worse off than if otherwise, all had cooperated [1–4]. Indirect reciprocity is suggested as a way out of this dilemma [3, 5–8], which can also bring insights into the evolution of morality [6, 7]. Most models of indirect reciprocity consider a simple strategic setting where individuals face a social dilemma, commonly modeled as a Prisoner’s Dilemma. Individuals decide upon their strategy in the social dilemma based on their opponent’s reputation. In turn, reputation is built based on the strategy of the individuals in the same social dilemma. This self-referential structure can give rise to some problems. The core of these problems relies on how to define good and bad. In the simplest indirect reciprocity model, only first-order moral assessment rules are allowed: For instance, an individual’s reputation is increased by cooperation, and it is decreased by defection [9, 10]. This leads to a situation where defection with someone with a bad reputation leads to a bad reputation. From a moral perspective, this does not make sense. Besides, this can lead to the instability of the dynamic [6]. To solve these problems, it is possible to consider second-order moral assessment rules [11–15]. In prescribing an individual’s reputation, besides its action, second-order rules also take the reputation of its opponent into account. However, this way opens the door to third-order and higher-order moral assessment rules [16–19], which require having information about the actions of the individuals further and further into the past [6, 19, 20]. This leads to a rapid increase in the number and complexity of moral assessment rules by going to higher-order rules, even when, as it is commonly assumed, moral assessment is reduced to a binary world of good and bad [6, 17–21]. In contrast to the premise of most models of indirect reciprocity, strategic interactions in many biological contexts are not simple. Individuals in populations often need to solve different strategic problems simultaneously. For instance, they may need to decide whether incur a cost for others to benefit, resolve conflicts to avoid mutual losses [22], coordinate their actions with others [22–25], for instance by deciding at what time or place engage in an activity, or they may need to choose partners and mates [26]. Importantly, the strategic structures of these diverse activities are generally distinct. While some entail a pure cost for the individual for the sake of others and thus constitute a social dilemma, such as donation of food or other resources, others can be mutually beneficial, such as conflict resolution or coordination in a group, for example for hunting [24, 25]. Arguably, some kinds of strategic complexity have been considered in evolutionary games. Examples include strategy dependent stochastic transitions between social dilemmas [27, 28], deterministic transitions between games [29, 30], the dynamics of two or more evolutionary games played in parallel, in one shot [31–35] (also coined multigames [33]), or repeated interactions [36], heterogeneity in the payoff structure of the games played by the individuals [37–40], the interaction of social dilemmas with signaling games [41–43], or the interaction between different social dilemmas in interacting heterogenous public goods games [44, 45]. However, despite their overwhelming prevalence in human and possibly other animal populations, the strategic complexity resulting from the interaction of games with different structures has remained poorly studied in evolutionary games. Here, I consider such a strategically complex situation where individuals face different strategic settings, each with a different set of actions and outcomes. In the language of game theory, this is to say individuals play different games with different strategies and payoffs. In such a context, the individuals’ strategic decisions in one game can depend on what happens in the other games. This provides a way to solve the self-referential problem in the models of indirect reciprocity, as it allows the reputation-building mechanism and the decision-making mechanism to occur at different levels, i.e., based on different strategic settings. As I show here, this observation can give rise to the evolution of a set of cooperation supporting moral norms. In our model, individuals play a Prisoner’s Dilemma, together with a second game, which I call game B, and is not necessarily a social dilemma. Individuals build a reputation based on their behavior in the Prisoner’s Dilemma and act based on this reputation in the game B. I consider a situation where both games are played with the same opponent or when the two games are played with different opponents. When game B is a pure dominance game with only one Nash equilibrium, such as a Prisoner’s Dilemma, the same problem incurred by indirect reciprocity models prevents the evolution of cooperation: Individuals are better off playing the Nash equilibrium irrespective of their opponents’ reputation. Consequently, the Nash equilibrium of the two-stage game can be decomposed into two simple Nash equilibria of the two composing games. The situation changes when game B has more than one equilibrium such that individuals can coordinate on a superior equilibrium, or avoid coordination failure, by taking the information about the strategy of their opponent into account in their game B strategic choices. In this case, in addition to the trivial defective Nash equilibria, a cooperative equilibrium exists in which coupling between games emerges and individuals decide upon their strategy in the second game based on their opponent’s strategy in the first game and cooperation in the social dilemma is sustained, although full cooperation often does not evolve. A static analysis of the games provides simple rules for the existence of a cooperative Nash equilibrium in the two-stage game. When game B is a coordination game, a cooperative pure strategy Nash equilibrium exists when the cost of cooperation is smaller than the excess payoff gained by coordinating on the superior equilibrium. This condition also ensures the existence of a cooperative fixed point in the evolutionary dynamics. When game B is a Snow Drift game, belonging to the anti-coordination class, a cooperative pure strategy Nash equilibrium exists when the cost of cooperation is smaller than the coordination asymmetry. By examining the evolutionary dynamics when the game B is a Stag Hunt game, I show that when game B belongs to the coordination class, such that it has two symmetric equilibrium, both a cooperative state where cooperators and defectors coexist, and a defective state, where defectors dominate are possible in a mixed population. However, the cooperative state has a small basin of attraction, and thus, convergence to such a state requires cooperation-favoring coordination norms to be encoded in the initial conditions. Furthermore, this cooperative state disappears in a structured population. The situation is different when game B belongs to the anti-coordination class, such that it possesses an asymmetric equilibrium. In this case, a symmetry-breaking phase transition exists above which the symmetry between cooperation and defection breaks. A set of behavioral rules emerges according to which cooperation stands out as a valuable trait, and individuals play softly with cooperators (i.e., they play strategies that gives a higher payoff to the opponent with cooperators). This leads to the evolution of cooperation. While the evolution of moral norms often lead to the coexistence of cooperative and defective strategies, in which cooperation maintained due to receiving a higher payoff from game B, depending on the strength of social dilemma and the structure of game B, moral norms can also fully suppress defection and give rise to full cooperation in the social dilemma in the stationary state of the dynamics. Importantly, this set of norms also promotes a more efficient allocation of resources in game B. This is particularly the reason why it evolves based on individuals’ self-interest. This observation appears to conform to the view that many aspects of moral systems do not necessarily require self-sacrifice but simply help to foster mutualistic cooperation and bring order and organization into societies [46–50]. In this regard, our analysis suggests that moral systems behave like a Trojan horse: Once established out of the individual’s self-interest, they also promote cooperation and self-sacrifice. Importantly, the conditions for the evolution of cooperation depend only on the cost of cooperation and are independent of other parameters of the Prisoner’s Dilemma. The fact that the evolution of moral norms only depends on the cost of cooperation and is independent of its benefit implies that even bad norms, which incur a pure collective cost, can evolve and promote organization and order. Analysis of the model in a structured population shows that population structure can remove the bistability of the system and ensures the evolution of cooperation favoring moral norms starting from all the initial population configurations when game B is an anti-coordination game. In contrast, in those simulations in which game B is a coordination game, we did not observe the evolution of cooperation. Furthermore, noise in inferring reputation can have a surprisingly positive effect on the evolution of a moral system in structured populations. However, this may not compensate for the loss cooperators experience due to a high recognition noise level. More ever, I show a very high level of recognition noise facilitates the evolution of cooperative behavior in the snow-drift game. This contrasts previous findings regarding the detrimental effect of population structure on the evolution of cooperation in the snow-drift game in simple strategic settings [51], and parallels some arguments regarding the beneficial effect of noise for biological functions [52–55]. The structure of the paper is as follows. First The model for both direct interactions and indirect interactions is introduced. Results Section begins by a Static analysis of the games and by deriving the Nash equilibria of the two-stage game shows that two classes of equilibria, the simple defective equilibria which are decomposable to two simple Nash equilibria of the composing games, and the cooperative equilibria where coupling between the games emerges, exists in the two-stage game, and derives simple Conditions for the existence of cooperative Nash equilibrium in terms of the cost of cooperation. Evolutionary dynamics: Mixed population studies the evolutionary dynamics in a mixed population and shows both defective fixed point where cooperation does not evolve and cooperative fixed points in which cooperation evolves exist. Fixed points of the evolutionary dynamics corresponding to the defective and cooperative equilibria are studied and by deriving the mixed strategy Nash equilibria corresponding to the fixed points, it is shown that similarly to the conditions for the existence of the cooperative Nash equilibria, The evolution of moral norms depends only on the cost of cooperation. By studying the Basin of attraction of the fixed points it is shown that the basin of attraction of the fixed points are dramatically smaller when game B is a coordination game compared to when it is an anti-coordination game, which points towards fundamentally different mechanisms underlying the evolution of cooperation in these two cases. Then by studying the Time evolution of the system when game B belongs to the anti-coordination class, it is shown that a set of cooperation favoring moral norms evolve in the course of evolution through a rapid dynamical transition due to the density-dependent selection of a costly cooperative trait. Evolution of moral norms in the direct interaction model and Evolution of moral norms in the reputation-based model further study the evolution of moral norms in the direct interaction and reputation-based model by simulations in finite populations and replicator dynamics for different archetypal games belonging to the anti-coordination class and shows that moral norms can lead to both cooperation in the Prisoner’s dilemma and a better allocation of resources and anti-coordination in game B. In Continuous variations of the structure of game B and the evolution of moral norms through a symmetry breaking phase transition by studying the continuous variation of the structure of game B the physics of the evolution of moral norms is studied and it is shown that moral norms evolve by a symmetry-breaking phase transition above which the symmetry between cooperation and defection breaks and a set of cooperation-favoring moral norms evolve. Finally, the model in a Structured population is studied and it is shown that while cooperation in a structured population does not evolve when game B is coordination game, it does evolve when game B is an anti-coordination game. Furthermore, it is shown that population structure removes the bistability of the system and ensures the evolution of moral norms starting from all the initial conditions. Finally, it is shown that recognition noise can facilitates the evolution of moral norms in a structured population. In the Discussion it is argued how the findings can shed light on the evolution of indirect reciprocity and the evolution of harmful norms, and how the mechanism underlying the evolution of moral norms relates to the evolution costly traits.

The model I begin by introducing two slightly different models. In the first model, the direct interaction model, information about the strategies of the individuals is acquired through direct observation. In this model, I consider a population of N individuals. At each time step, individuals are randomly paired to interact. Each pair of individuals play a Prisoner’s Dilemma (PD), followed by a second game. The second game is a two-person, two-strategy symmetric game, which I call game B. I call the two possible strategies of game B, down (d), and up (u) strategies. The strategy of an individual in game B is a function of its opponent’s strategy in the first game. Thus, the strategy of an individual can be denoted by a sequence of three letters abc. Here, the first letter is the individual’s strategy in the PD and can be either C (cooperation) or D (defection). The second letter is the individual’s strategy in the game B if its opponent cooperates in the PD, and the last letter is the individual’s strategy in the game B if its opponent defects in the PD. Clearly, we have b, c ∈ {u, d}. For example, a possible strategy is to cooperate in the PD, play d if the opponent cooperates, and play u if the opponent defects. I denote such a strategy by Cdu. While in the direct interaction model, individuals play both their games with the same opponent, I also consider a second model, the reputation-based model, where individuals play their two games with different opponents. In this model, individuals are randomly paired to play a PD at each time step. After this, the interaction ends, and individuals meet another randomly chosen individual to play their second game. Under this scenario, individuals do not observe their opponent’s strategy in the PD. Instead, I assume individuals have a reputation of being cooperator or defector, on which their opponent’s decision in the second game is based. For example, an individual with the strategy Cdu, cooperates in the PD, plays d if it perceives its opponent to be a cooperator, and plays u if it perceives its opponent to be a defector. To model reputation, I assume with a probability 1 − η individuals are informed about the PD-strategy of their opponent, and with probability η they make an error in inferring the PD-strategy of their opponent. η can be considered as a measure of noise in inferring the reputations. I note that, for η = 0, the dynamics of the two models are mathematically similar. For the evolutionary dynamics, I assume individuals gather payoff according to the payoff structure of the games and reproduce with a probability proportional to their payoff. Offspring inherit the strategy of their parent. However, with probability ν a mutation occurs, in which case the strategy of the offspring is set to another randomly chosen strategy. I will also consider a structured population. While in a mixed population, individuals randomly meet to interact, in a structured population, individuals reside on a network and interact with their neighbors. That is, each individual derives payoffs by playing its two games with all its neighbors. For the evolutionary dynamics, I consider an imitation rule, in which individuals update their strategy in each evolutionary step by imitating an individual’s strategy in their extended neighborhood (composed of the individual and its neighbors) with a probability proportional to its payoff, subject to mutations. For the population network, I consider a first nearest neighbor square lattice with von Neumann connectivity and periodic boundaries. The payoff values of the games are as follow. In the PD, individuals can either cooperate or defect. If both cooperate, both get a payoff R (reward), and if both defect, both get a payoff P (punishment). If an individual cooperates while its opponent defects, the cooperator gets a payoff S (sucker’s payoff), while its opponent gets a payoff T (temptation). For a Prisoner’s Dilemma, we have S < P < R < T with T + S < 2R. For game B, I show the payoff of mutual down by R B , and the payoff of mutual up by P B . If an individual plays up while their opponent plays down, the up-player gets T B , and the down-player gets S B . I will analyze the model for different structures for game B. Here I consider only symmetric games. Symmetric two-player two-strategy games can be divided into three classes: pure dominance (such as Prisoner’s dilemma), coordination games (such as Stag Hunt game), and anti-coordination games (such as Snow Drift game) [56]. As mentioned in the introduction, when game B is a pure dominance game, such as a social dilemma, cooperation in the two-stage game does not evolve, while cooperation can evolve when game B is a coordination game or an anti-coordination game. As for the anti-coordination class, I consider cases where game B is a Snow Drift (SD) game (also known as the Hawk-Dove or Chicken game) [2], the Battle of the Sexes (BS), and the Leader game. Together with the Prisoner’s Dilemma, these games are coined as four archetypal two-person, two-strategy games [57]. I will also consider a case where game B is a Stag Hunt (SH) game, which belongs to the coordination class [56]. Furthermore, I examine the dependence of the results on the continuous variation of the structure of the game B. All the anti-coordination games mentioned above have an asymmetric Nash equilibrium. In the Nash equilibrium, one of the strategies is superior in the sense that it leads to a higher payoff, leaving the opponent with a lower payoff. I call such a superior strategy the hard strategy, and the inferior strategy, which leads to a lower payoff in equilibrium, is called the soft strategy. When using the three archetypal games, I take the soft strategy to be the same as down and the hard strategy to be the same as up. The base payoff values used in this study (unless otherwise indicated) are presented in Table 1. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Table 1. Base payoff values. https://doi.org/10.1371/journal.pcbi.1010429.t001

Discussion We have studied the evolution of strategies in a complex strategic setting, where individuals in a population play different games and base their strategy in a game on what happens in another game. By considering a situation where two interacting games exist, a Prisoner’s Dilemma followed by a second game, we have seen that as long as the second game belongs to the coordination or anti-coordination class, the Nash equilibrium of the two-stage game can be of two types: defective equilibria, where the Nash equilibria of each of the composing games are played, and cooperative equilibria, in which coupling between games emerge and give rise to a new class of Nash equilibria not reducible to the Nash equilibria of the composing games. For the cooperative equilibrium to exist, the cost of cooperation should be smaller than a value determined by the structure of the second game. A similar condition ensures the existence of a cooperative fixed point in the evolutionary dynamics. Investigation of the evolutionary dynamics shows that a cooperative fixed point exists in a mixed population for both coordination and anti-coordination games. However, while in the former case, the cooperative fixed point has a small basin of attraction and disappears in a structured population, in the latter, a cooperative fixed point in which a set of cooperation supporting moral norms emerges and supports cooperation can evolve starting from a rather broad range of initial conditions. The evolution of moral norms, in this case, originates from the fact that it is in the individuals’ best interest to take the information about what happens in the social dilemma into account when making strategic choices in a second game to better anti-coordinate in the second game. Consequently, in the course of evolution, a set of cooperation supporting moral norms emerges based on the individual’s self-interest. This appears to provide a possible mechanism for the evolution of morality in a biological population composed of self-interested individuals with simple cognitive abilities. Importantly, population structure facilitates the evolution of a moral system by removing the bistability of the system and ensuring a cooperative state to flourish starting from all the initial conditions when the second game is an anti-coordination game. Analysis of the model in a structured population shows that noise in recognition can be beneficial for the evolution of a moral system in structured populations. Nevertheless, recognition noise also limits cooperators’ ability to benefit from stronger moral norms, and thus, adversely affects cooperation. Finally, our analysis reveals in a complex strategic setting, very high levels of recognition noise facilitate the evolution of cooperative behavior in the Snow Drift game in structured populations. This shows, in contrast to what is the case in a simple strategic setting [51], network structure can be beneficial for the evolution of cooperation in the Snow Drift game. Furthermore, this provides another case for the surprisingly beneficial role that noise may play for biological functions [52–55]. Our findings provide new insights into the evolution of indirect reciprocity. By considering a simple strategic setting, namely one in which individuals only can play a social dilemma, models of indirect reciprocity have shown that specific moral rules can support an evolutionary stable cooperative state. However, the simplicity of the strategic setting requires the moral assessment module and action module to occur in the same context, which is typically a social dilemma game that individuals play. This self-referential structure can destabilize the dynamics. To solve this problem, the theory often appeal to higher-order and complex moral assessment rules. In addition to the lack of a natural mechanism to break the chain of higher-order rules, this requires a relatively high cognitive ability and a large amount of information about the past actions of the individuals in moral assessment [19], which appears to limit its applicability [20]. Furthermore, the dichotomy of moral assessment module and action module, commonly incorporated in many models of indirect reciprocity, can give rise to severe problems, when instead of public information, individuals have private information about the reputation of others [7, 59–62]. In this case, a punishment dilemma can arise: individuals may have different beliefs about the reputation of others, and thus, disagree as to what is a justified punishment [5, 59]. In contrast, as I have shown, the introduction of interaction between games circumvents these problems and leads to a simple dynamical mechanism for the evolution of a set of cooperation supporting moral norms. In this regard, in a strategically complex context, it is not necessary to define good and bad a priori to the dynamics of the system. This avoids the punishment dilemma when information is private. Nor it is necessary to define different moral assessment rules [12–14, 17, 18] and search for efficient ones [17–19, 21, 63]. Rather, the dynamics self-organizes into a symmetry broken cooperative phase where the symmetry between cooperation and defection breaks. A set of cooperation supporting moral norms evolves and costly cooperation emerges as a morally valuable or “good” trait due to a purely dynamical phenomenon and as a result of a symmetry-breaking phase transition. As our analysis shows, a moral system not only promotes cooperation in a social dilemma, but it also increases soft strategies which can be considered as a more rational form of cooperative behavior in a second game, a strategic setting which may be a social dilemma (as in the case of SD) or may not be a social dilemma. By considering three archetypal games, and continuous variations of the structure of the second game, I have shown this is the case for a broad range of strategic settings. In this sense, a moral system not only works to promote cooperation, but it also helps to solve coordination problems and help an efficient allocation of roles and resources. This finding seems to conform to many stylized facts about moral systems. For instance, while some moral values encourage self-sacrificing and other-regarding behavior [7, 46, 64], many other aspects of moral systems do not seem to go against individuals’ self-interest, but encourage mutually beneficial behaviors, such as mutualistic cooperation [46–50], or conflict resolution [47, 49]. Fairness, loyalty, courage, respecting others, cherishing friendship, working together, and deferring to superiors are examples of such mutualistic moral values. Based on these observations, it is suggested promoting mutually beneficial behavior can provide yet another explanation for the evolution of morality [48–50]. Importantly, in our model, this second role is what makes a moral system evolvable based solely on the individuals’ self-interest. In other words, the positive role of a moral system in bringing order and organization is beneficial at both the individual and group levels. This makes adherence to a moral system beneficial on an individual level and helps its evolution in a simple dynamical way. Interestingly, this aspect of a moral system acts like a Trojan Horse: Once established due to its organizing role, it also suppresses anti-social behavior and promotes cooperation and self-sacrifice. Another aspect of the theory developed here is that the cost of cooperation can be considered as a cost paid by individuals to reach a high moral status to benefit from favorable encounters in interactions that do not involve a strict social dilemma. Interestingly, the model shows that the stronger the social dilemma and the higher the cost of cooperation, a stronger set of cooperation supporting moral norms emerges, as the likelihood that cooperators receive a favorable encounter in the game B increases with increasing the strength of the social dilemma (see Fig G in S1 Text for a mixed population, and Fig T in S1 Text for a structured population). Although this may not fully compensate for the higher cost of cooperation cooperators pay in stronger dilemmas, it partly alleviates cooperators’ loss of payoff and helps the evolution of cooperation when the cost of cooperation is high. Some empirical evidence has shown that costly cooperative traits can give cooperators an advantage over a diverse set of strategic contexts, such as coordination, partner choice, and conflict resolution [23–25, 65]. The prevalent theoretical understanding of these contexts, in the framework of costly signaling theory, is that the costly cooperative traits can function as an honest signal of quality based on which cooperators might receive more favorable interactions [24, 65]. More recently, an alternative theory has pointed out a simple dynamical phenomenon resulting from the density-dependent selection that can explain why cooperative traits can bring more favorable strategic responses [41]. This theory shows that the very fact that a trait is costly can lead to its scarcity, which in turn can lead to the evolution of favorable strategic responses, as such strategies do not impose a high cost on their bearer. A similar density-dependent selection can give rise to the evolution of cooperation in costly public goods [45], or consistent cooperative personalities in multistage public goods [66]. The models studied here show that a similar dynamical phenomenon can underlie the evolution of moral norms in complex strategic settings. This is the case because strategies that play softly with cooperators do not impose a high cost on their bearers due to the scarcity of costly cooperative traits. This can lead to the evolution of social norms that favor cooperative behavior over diverse issues such as conflict resolution and coordination. Counter-intuitively, once such cooperation-favoring norms evolve, the dynamics can get fixated in a cooperative state where the very tenet underlying the virtue of cooperative traits, their scarcity, get lost in a cooperative fest. This theory thus can provide an alternative explanation for the prevalence of cooperation-favoring norms in human societies, usually looked at through the lens of costly signaling theory. The density-dependent selection underlying the evolution of moral norms has a surprising consequence: harmful social norms are just as likely to evolve as socially beneficial ones. A large body of empirical evidence has documented the existence of harmful social norms [67, 68]. Such harmful norms abound in cultures of honor, ranging from honor killing [67, 69], to pious and harmful cultural practices [68] and irrational and severe punishment [70, 71]. Inefficient gift-giving is suggested as another example of such bad social norms involving a collective loss [71]. While it is argued that reputation and reciprocity play an important role in the evolution of such harmful norms [67, 72], given their collective cost, the evolution and persistence of such harmful norms seem a puzzle. Our theory provides a simple explanation for the evolution of such detrimental norms: the cost of norms, not their benefit, determines their evolution. In other words, moral norms need to be costly for the individual but not necessarily beneficial for the group. This fact can give rise to bad norms which incur a collective cost. Surprisingly, such costly norms are just as effective as socially beneficial norms in promoting order and organization, and this phenomenon underlies their evolvability within the framework developed here. According to this viewpoint, the key to the puzzle of the evolution of harmful social norms does not rely on their potential advantage. Instead, their evolution is a consequence of the evolutionary process underlying the evolution of moral systems. Intuitively, costly traits provide a density-dependent mechanism for coordination and more efficient allocation of resources, and this fact underlies the evolution of norms that prescribe differing to such costly traits. This, counter-intuitively, can lead to an equilibrium state that costly traits are neither costly nor rare anymore because of the payoff they accrue due to favorable strategic responses. This framework appears to explain why many moral systems incorporate both socially beneficial and harmful but often individually costly elements. Furthermore, the positive role that our theory suggests that costly norms play in bringing order and organization appears to conform to the fact that culture of honors often originate and persist in law-less environments and play a crucial role in stabilizing societies in the absence of law-enforcement organizations.

Methods The replicator-mutator dynamics The model can be solved in terms of the discrete-time replicator-mutator equation, which reads as follows: (1) Here, ρ x is the density of strategy x, π y is the expected payoff of strategy y, is the mean payoff, and ν x,y is the mutation rate from strategy y to the strategy x. This can be written as: (2) The payoff of an strategy can be written as follows. First I define: (3) Here, the first letter in the indices shows the strategy in the PD, and s(C) (s(D)), is the strategy in the second game against a cooperator (defector). That is, for example, ρ C,u(C) is the density of those individuals who cooperate in the PD and play the up strategy with cooperators. Besides, in the following, I use ρ C and ρ D for the total density of those individuals who, respectively, cooperate and defect in the PD. That is, ρ C = ρ Cuu + ρ Cud + ρ Cdu + ρ Cdd and ρ D = ρ Duu + ρ Dud + ρ Ddu + ρ Ddd . Given these definitions, the payoffs of different strategies, in the first model can be written as follows: (4) Here the first two terms in each expression are the payoffs from the PD, and the last four terms are the payoff from game B. The validity of these expressions can be checked by enumerating all the possible strategies that a focal individual can play with. For example, the third term in the expression for π Cuu can be written by noting that a focal Cuu player, meets an individual of type C, u(C) with probability ρ C,u(C) . In this interaction, the focal individual plays u and the opponent plays u, leading to a payoff of P B for the focal individual. Using similar arguments, it is possible to drive expressions for the payoff of different strategies in the reputation-based model. See the Supporting Information Text, S. 2 for details. Mixed strategy Nash equilibria Cooperative mixed strategy Nash equilibria when game B is a coordination game. A mixed strategy is a set of probabilities, {x Cuu , .., x Ddd }, such that a strategy i is played with probability x i . The support of a mixed strategy is the set of all strategies which are played with nonzero probability. A mixed strategy Nash equilibrium is defined as a set of two mixed strategies, (s, s′) in which each strategy is the best response to the other strategy: s = BR(s′) and s′ = BR(s) [73]. This condition is achieved for a mixed strategy s if the payoff of all the strategies in the support of a mixed strategy is the same, and no other strategy outside of the support gives a higher payoff against the mixed strategy [73]. The first criteria can be satisfied by solving a set of linear equations to achieve indifference of each player over the support of their mixed strategy. To see this, consider the mixed strategy corresponding to the cooperative fixed point I of the evolutionary dynamics when game B is the Stag Hunt game. The expected payoff of the strategy Cdd against a mixed strategy, (x Cdd , c Ddu ) is (R + R B )x Cdd + (S + R B )x Ddu and the expected payoff of Ddu is equal to (T + R B )x Cdd + (P + S B )x Ddu . Equaling these payoffs and setting the normalization condition, x Cdd + x Ddu = 1, gives a set of two equations which can be solved to give: (5) It is also easy to check that within the general parametrization of the Stag Hunt game, no other strategy gives a higher payoff than Cdd and Ddu against this mixed strategy (the highest payoff outside of the support is reached by Ddd which is equal to that reached by Ddu and Cdd). Using similar steps, it is possible to derive the mixed strategy corresponding to the fixed point II of the evolutionary dynamics when game B is a Stag Hunt game as follows: (6) Cooperative mixed strategy Nash equilibria when game B is an anti-coordination game. The mixed strategy Nash equilibria corresponding to the cooperative fixed point of the evolutionary dynamics when game B is an anti-coordination game involves the strategies Cuu, Cdu, Ddu, and Ddd, and can be derived using similar steps. Using the helping game version of the Prisoner’s dilemma, this mixed strategy is given by the following expression: (7) These expressions describe the cooperative fixed point of the evolutionary dynamics for all the three games belonging to the anti-coordination class considered. Using the general formulation of the Prisoner’s Dilemma, it turns out that the fixed point only depends on two combinations, T − R and P − S, which are equal to the cost of cooperation in the Helping game version of the Prisoner’s dilemma. This implies that the condition for the evolution of cooperation only depends on the parameters of the Prisoner’s dilemma through the cost of cooperation. Fully cooperative mixed strategy Nash equilibrium. As we have seen, when game B is a Snow Drift game, for small cost of cooperation, the evolutionary dynamics of the two-stage game has a fully cooperative fixed point which is composed only of the cooperative strategies. This fixed point corresponds to a fully cooperative mixed strategy Nash equilibrium. To derive this equilibrium, consider the mixed strategy defined by the probabilities, x Cuu , x Cud , x Cdu , x Cdd , and zero probability of playing the defective strategies. The expected payoff of Cuu and Cud is equal to x Cuu (R + P B ) + x Cud (R + P B ) + x Cdu (R + T B ) + x Cdd (R + T B ) and the expected payoffs of Cdu and Cdd is equal to x Cuu (R + S B ) + x Cud (R + S B )+ x Cdu (R + R B )+ x Cdd (R + R B ). Equating these payoffs, subject to the normalization condition x Cuu + x Cdu + x Cud + x Cdd = 1, we derive for the probabilities: (8) For this strategy profile to be a Nash equilibrium, the payoff of the strategies in the support should be at least as high as the payoff of the all the strategies outside the support of the mixed strategy. The payoffs of Duu and Dud strategies against this mixed strategy equals x Cuu (T + S B ) + x Cud (T + T B ) + x Cdu (T + P B ) + x Cdd (T + T B ), and the payoffs of Ddu and Ddd equals x Cuu (T + S B ) + x Cud (T + R B ) + x Cdu (T + S B ) + x Cdd (T + R B ). The payoff of all the strategies in the support, which are now equalized using Eq 8, is equal to . Requiring the payoffs of the defective strategies to be larger than this value, (9) subject to Eq 8, and using the payoff values of the Snow Drift game, we arrive at the condition: (10) Eqs 8 and 10 define a set of two equations that give infinitely many fully cooperative mixed strategy Nash equilibria. By using Eq 10, it can be seen that such fully cooperative mixed strategy Nash equilibria exist only if the cost of cooperation is smaller than 2/3, which agrees with the results from the replicator-mutator dynamics presented in Fig 4. It is possible to derive conditional expressions for the existence of a fully cooperative Nash equilibrium for a general form of the game B belonging to the anti-coordination class. By using such expressions, it is possible to show that no fully cooperative mixed strategy Nash equilibrium exists when the game B is the leader or the Battle of the Sexes, given by the payoff values presented in Table 1. Simulations and numerical solutions Numerical solutions result from numerically solving the replicator-mutator dynamics. Simulations are performed based on the model definition. Matlab codes used in simulations and numerical solutions are given in the Supporting Information Text, S. 9. The base payoff values used in this study (unless otherwise stated) are presented in Table 1. See Supporting Information Text for more details on simulations and analytical calculation.

[END]
---
[1] Url: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1010429

Published and (C) by PLOS One
Content appears here under this condition or license: Creative Commons - Attribution BY 4.0.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/

You are viewing proxied material from magical.fish. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.