Making it Real: The PD when Death is on the Line
This is a rewrite of my masters dissertation from the University of Connecticut, 2001.
The Prisoners’ Dilemma has been modeled on many computer simulations in an attempt to determine under what conditions cooperation might arise. The simulation employed here adds a dimension to the Prisoners’ Dilemma by allowing whole populations to compete against each other under more realistic conditions. The rewards and penalties are not merely points accumulated, but are translated into real benefits whereby members of a population can have offspring or die prematurely from starvation depending on how they perform against competing strategies. Each player in a simulation starts out with a base amount of ‘life points.’(1) From there the simulation is played a number of times (iterations) and the players allowed to interact with each other. With each iteration, two players are selected at random from the total population and allowed to engage in one Prisoners’ Dilemma game. The results are added to or subtracted from each player’s life points. After accumulating a certain number of life points, a player is allowed to have an offspring which inherits his parent’s strategy. Lose too many points and the player dies.
Figure 1. After 100 iterations between Always Cooperate, Always Defect and Tit For Tat. All-D is clearly dominating the population.
Figure 2. Cunning Neighbor enters the population at iteration 101. After 200 iterations Cunning Neighbor has taken the number two place over TFT and All-C.
Figure 3. After 1,000 iterations CN is beginning to overtake All-D.
Figure 4. After 10,000 iterations CN has established an overwhelming dominance of the population.
In a straight contest between TFT and CN, CN proved to be very effective (See figure 5.) The balance quickly tipped in CN’s favor and TFT was driven to a minority by iteration 200. CN continued to overwhelm the population but was unable to completely eliminate TFT(4). After 1,000 iterations, CN represented 338 members to TFT’s three.
Good Neighbor (GN) is a strategy that always cooperates with other neighbors, just like CN. When it comes to outsiders, a GN will try to be nice at first. But unlike TFT that must learn on a case by case basis who is friendly and who is nasty, a GN can make a judgment about the outside population as a whole. He does this by keeping track of how many times he receives the prizes, R, S, T and P. Lots of R's and T's indicate a friendly environment. Lot's of S's and P's indicate a hostile environment. (See Appendix A. Good Neighbor for a discussion on how GN functions.)
Figure 6. Good Neighbor, Always Defect and Tit For Tat compete.
In a contest between GN and All-D, Good Neighbor will not do as well as All-D, but will be able to survive (See Figure 7.) Each Good Neighbor begins by being friendly, but immediately switches to always defecting.
Figure 7. Good Neighbor and Always Defect.
Wise Neighbor (WN) is like GN, except that he has the ability to pass on experiences to the rest of its group (See Appendix B. Wise Neighbor for a discussion on how the simulation calculates Wise Neighbor’s responses.) Experiences with the outside world are recorded for the whole group and not just for each individual. In a population of five Wise Neighbors, the first four can learn about the outside world. When WN number five has his first encounter with an outsider he will cooperate or defect based on how well the other four have fared. This overcomes the problem of isolation and youthful naiveté of GN. With GN, each neighbor must learn for himself what the outside environment is like. New entities born into the WN community can learn from the past experiences of the group. Knowledge can be encoded in the wisdom of the group, in other words.
Figure 8. Wise Neighbor and Always Defect
Wise Neighbor does very well against Always Defect (See figure 8.) It very quickly detects the hostile environment and defends against it. It is also able to transmit this information on to other members of the group, including newborn members, thereby preparing members of the next generation for their first encounters with the outside world.
Not surprisingly, Cunning Neighbor does very well against individual strategies like Tit for Tat. Even when a game is played for one hundred iterations and has grown in size from the original players, CN is able to infiltrate it and quickly dominate. By 10,000 iterations it represents 90 percent of the population, up from 12 percent when it first entered the population.
Good Neighbor and Wise Neighbor do poorer. They are able to detect a hostile environment and defend themselves against it. They are also able to live peacefully in a nice environment. The problem with these strategies is in an ambiguous environment. In a three way game with Cunning Neighbor and All-C, it takes a while for GN or WN to decide which strategy to pursue: Cooperate or Defect. Since these strategies shun the Temptation, they bounce back and forth between Cooperating and Defecting until the outside environment becomes mostly hostile at which point they fall into a pattern of All-D, even against the remaining nice outsiders.
Wise Neighbor is better off than Good Neighbor. Once one WN determines that the outside environment is hostile, every other WN will defect. With GN each neighbor will cooperate first and then decide what the outside environment is like.
Figure 10. Cunning Neighbor, Always Cooperate and Good Neighbor.
Interestingly, Cunning Neighbor proved to be a highly effective strategy as well as being very information efficient. All it requires for a CN to decide on a strategy is one piece of information: Friend or Foe? Tit For Tat, on the other hand, requires a lot of information processing. Since the simulation operates in a population that can easily grow into the hundreds of thousands, each player must have a photographic memory of every other player and how that player responded in the past.
The simulation program creates a record for each game played. It records the ID's of the two players, the iteration of the game where this encounter takes place and the decisions of the two players. This is a very small record stored in a database. The next time a player like Tit For Tat or Tit for Two Tats plays, the simulation must search that database looking for records of an encounter between these two players in the past. If a simulation has run for ten or twenty thousand times, it must search ten or twenty thousand records. As simulations get bigger, the amount of time to execute grows longer with the increased risk of mistaken identify causing an inappropriate response. With an empty database, the best time for 10,000 iterations was about one minute. With a larger database containing a much bigger history, the maximum run time was over an hour and ten minutes(5).
From the standpoint of information processing this is significant. It also suggests that a strategy that requires personal recognition is costly in information processing terms. Something like Cunning Neighbor is much easier to encode. All that is needed is a recognition system based on the most general of criteria. A secret handshake, tell tale sign or Shibboleth is much easier to process than having to remember every past encounter with every potential partner.
For the other group strategies, a slightly more sophisticated technique is needed, but not nearly as data processing intensive as Tit For Tat. What is needed is the same recognition mechanism, plus a general memory of the overall responses of the opponents as a whole. Instead of individual opponents, opponents are lumped together into a collective 'them' and an overall impression formed.
Cunning Neighbor is deterministic: Us against Them. The other strategies develop prejudices and can become biased. They react to new outsiders based entirely on how they have viewed them in the past. If members of a certain outside group tend to be friendly, future encounters will be treated favorably. If not, a ‘once burned, twice learned’ approach will be used instead.
Has this proven selection of group strategies? Possibly, but only in that loyalty to ones group can ultimately increase the fitness of the individual. Loyalty to ones group makes it possible for the individual to exploit that group. Since ‘fitness’ is a measure of ‘exploitation,’ this is a well understood and accepted principal of evolution(6). By identifying with ones group and working toward the good of that group one can exploit the resources available only in association with that group. The memory of past interactions can play a powerful role in shaping future decisions. Not just regarding interactions with ‘outsiders’ but those within ones own group as well. You do not cheat your neighbor just because he is your neighbor, but because he will remember and that will cloud his interactions with you in the future. None the less in different circumstances you may try to get more than you deserve in the hopes that you can get away with it. This balancing act between cooperation and self indulgence, between being nice and trying to get away with something, forms the basis of civilization’s teeter tooter existence.
Furthermore, a group loyal strategy is still only one strategy in the toolkit of individual survival. Sometimes an individual might be loyal to their group and sometimes he may act with total selfishness. It all depends on what’s appropriate and what that individual thinks will work at that instant. Each is a possible tool of interaction which can be employed equally as is necessary. Civilizations might be noble or they might be savage. Or they might just be the collection of both noble and savage behaviors that have come together and seem to work at this moment in history.
The balancing act between altruistic and selfish impulses swings back and forth. Too selfish and a social group falls apart due to a lack of trust. Too trusting and it becomes vulnerable to outside invasion. This balance of selfish and altruistic must self regulate between set extremes to persist.
After all, every culture has its myths of man’s higher and lower natures struggling with each other, good vs. evil, angels and demons, gods pushing stones up impossible mountains but never giving up, neither side relenting or overcoming the other, nor ever ceasing the struggle. Humanity is forever in the grips of his internal contradictions. For in both directions lies destruction.
In this paper I have tried to layout some conditions under which members of a group, a tribe or a village may form group identity and a loosely formed, though delicate, cooperation as well as a framework of interaction with outside groups. This may help illuminate, if not explain, the checkered past of human interactions within our species.
The tools of genetic behavior provided by evolution are varied and contradictory. What makes sense in one interaction might not in another. What may be considered savage in one context is perfectly acceptable in another, even if justifying that savagery requires belief systems beyond belief.
Good neighbor can recognize members of its neighborhood and automatically cooperates with others of its own kind. Members are entered into the simulation in groups which are flagged as members of the same group. When a Good Neighbor interacts with outsiders, he cooperates first and then subsequently relies upon his memory of past interactions. GN’s memory consists of four counters that record the frequency that it has received each of the four possible payoffs. These counters are indicated as f(R), f(S), f(T) and f(P). The GN will cooperate in this case.
In the case of wise neighbor, counters are kept for how well the group fares against outsiders. Information is shared, so it is not recorded at the individual level, but at the group level. There are four counters for all of the members of one group of WN’s. At the beginning of the simulation the counters are all set to zero. When the first member of the group encounters an outsider, it cooperates if f(R)+f(T) is greater than or equal to f(S)+f(P). Then the outcome is counted. Suppose that the outsider also cooperated. A 1 is added to the counter for f(R), the reward for cooperating. The next time a member of this group encounters an outsider, it can look at the experience of the group as a whole. So if a different member of the group encounters an outsider, it does not have to rely solely on its own experience. The group has already had one positive experience with outsiders, so this entity will cooperate in the new situation: f(R)+f(T) = 1, f(S)+f(P) = 0.
1988 The Further Evolution of Cooperation. Science, Vol 242, p. 9 December
1984 The Evolution of Cooperation. Basic Books, A subsidiary of Perseus
Barkow, Jerome H, Leda Cosmides & John Tooby
1992 The Adapted Mind. Oxford University Press, New York.
Bendor, Jonathan, Roderick M. Kramer and Suzanne Stout
1991 When in Doubt… Cooperation in a Noisy Prisoner’s Dilemma. Journal of
Conflict Resolution, Vol 35 No. 4, December 1991, p. 691-719.
1993 Uncertainty and the Evolution of Cooperation. Journal of Conflict
Resolution, Vol 37, no. 4, December 1993, p 709-734.
1996 Evolutionary Stability in the Finitely Repeated Prisoner’s Dilemma Game.
Journal of Economic Theory, Vol. 68, p. 234-248.
1995 Prisoner's Dilemma Popularized: Game Theory and Ethical Progress.
Dialogue (Waterloo, Ont.), v. 34 (Spring '95) p. 295-304.
1998 Prisoner’s Dilemma.
Dixit, Avinash K. and Barry J. Nalebuff
1991 Thinking Strategically. The Competitive Edge in Business, Politics,
and Everyday Life. W.W.Norton and Co., New York.
1988 Passions Within Reason: The Strategic Role of the Emotions.
Norton, New York.
Houston, Alasdair I.
1993 Mobility Limits Cooperation. TREE vol. 8, no. 6, June 1993.
Linster, Bruce G.
1992 Evolutionary Stability in the Infinitely Repeated Prisoners' Dilemma
played by two-state Moore machines. Southern Economic Journal,
v. 58 (Apr.'92) p. 880-903.
Lloyd, Alun L.
1995 Computing Bouts of the Prisoner’s Dilemma. Scientific American, June
1996 Nucleus and Shield: The Evolution of Social Structure in the Iterated
Prisoner’s Dilemma. American Sociological Review, 1996, Vol. 61
2001 The Prisoner’s Dilemma Where Death is on the Line
1990 The Inapplicability of Evolutionarily Stable Strategy to the Prisoner’s
Dilemma. The British Journal for the Philosophy of Science, v. 41
(Dec. '90) p. 461-72.
Macy, Michael W. and John Skvoretz
1998 The Evolution of Trust and Cooperation Between Strangers: A
Computational Model. American Sociological Review, 1998, Vol. 63
Moore, F. C. T. (Francis Charles Timothy)
1994 Taking the Sting out of the Prisoner's Dilemma. The Philosophical
Quarterly, v. 44 (Apr. '94) p. 223-33.
Nowak, Martin and Karl Sigmund
1993 Chaos and the Evolution of Cooperation. Evolution, Vol 90. pp.
5091-5094, June 1993.
1988 More Realism in the Prisoner’s Dilemma. Journal of Conflict Resolution,
Vol 32 No. 2, June 1988, p. 402-408.
Wu, Jianzhong and Robert Axelrod
1995 How to Cope with Noise in the Iterated Prisoner’s Dilemma. Journal
of Conflict Resolution, Vol. 39, No. 1, March 1995, p. 183-189.