A blog about problems in the field of psychology and attempts to fix them.

Sunday, April 8, 2012

Verbal Behavior, the Weather Man, and the Fundamental Lie of Professional Poker

I am reading Skinner's Verbal Behavior for the first time with my undergraduate class. It is amazingly good. The only criticism I would make thus far is that it is tinted throughout with the quirkiness of Skinner's particular brand of behaviorism, which is to be expected. Anyone who is trying to do language in an embodied or ecological context, who hasn't read this book and picked out the important points, should do so post haste. The next time I go through the book, a blog book-club will definitely be in order.

One question that my students raised early on struck at a fundamentally important point in modern intellectual development. What, they asked, does Skinner mean by:
Our basic datum is not the occurrence of a given response as such, but the probability that it will occur at a given time.
The idea that science is interested in probability, and not certainty, is still foreign to most people. However, it is a crucial idea, that permeates our modern world. Two great examples are found in the attitude of the professional weatherman and the professional poker player, both of which are poorly understood.

Evaluating the Weatherman
Most people have a very poor understanding of the weather game. They think that if the chance of rain is above 50%, then it should rain, and if the chance of rain is below 50% it should not rain. If the weatherman says there is a 25% chance of rain, and it rains, the average citizen curses and declare that the weatherman is bad at his job. But, of course, that type of evaluation does not judge the weatherman fairly. In fact, it should rain pretty often when the weatherman predicts a 25% chance of rain. If we look over a great many days, and find that it rains one out of four days in which a 25% chance is predicted, one out ten days in which a 10% chance is predicted, etc., then our weatherman is operating with perfect accuracy.

That is, the weatherman's job is not about telling you if it will or will not rain on an individual day. The weatherman's job is about telling you a probability that it will rain. If we look over time, the probability converts to a frequency, and long-term accuracy can be easily assessed.

Of course, there might be other things about your weatherman that make you mad, but accuracy is probably not the issue. (I for one, wish the weatherman would give a higher percentage of predictions at the extremes of the probability range.)

The Fundamental Lie of Professional Poker
One reason that poker is such a great game is that it is all about probability - of the cards, and of the actions of the other players. (For simplicity below, I will stick to the probability of the cards.) You make money in poker by acting, with respect to the probabilities, better than your opponents. The central role of probability is what makes it so hard to learn how to play poker well - the surface level operant conditioning effects are unlikely to lead to good play.

Let's talk learning theory:
  1. Winning a huge pot in a stressful situation is amazingly reinforcing - it makes the behavior that preceded the victory more likely in the future. 
  2. Losing a huge pot is not usually as punishing, because only a fraction of the money in the pot was yours.
  3. Variable  ratio reinforcement is very resistant to extinction. That is, if you develop a behavior in a context in which you cannot be sure how many times you have to do a given behavior before it gets reinforced, then you will persist for long periods in which no positive outcome occurs. 
Let's add some cognitive psychology:
  1. Most people think they are trying to predict which card will come next.
  2. Thus, people believe they have made the correct decision if the outcome is good, and the wrong decision if the outcome is bad. 
  3. When people win, they feel like they won the whole pot of money, but when they lose, they only feel like they lost the part they put in.
These principles, which make poker so alluring, will develop money-losing habits in most players. Learning theory: The wrong decision will often get reinforced, and the pattern of reinforcement make it very difficult correct course. Cognitive psychology: People don't think properly about their goals in the game, and so do not recognize a bad decision when they make one (not to mention the effects of cognitive dissonance, self-serving bias, etc.). To illustrate, let us examine scenario that would be common in 'Texas holdem' poker:
Each player has 2 unique cards, which the other player cannot see. There are 4 cards face up on the board, and one more card will be revealed. Let us say that Player A has three-of-a-kind, a very good hand. Player B has nothing at the moment, except four cards of the same suit and two high cards. If another card of the same suit comes up, then Player B will have five of the same suit, a flush, which beats Player A. In addition to the real scenarios in which she wins, Player B wrongly believes that she can win if she pairs either high card. Due to whatever circumstances have occurred, a large pot has been built up, let us say $6,500. Player A, who is currently ahead, pushes his last $2,500 into the middle. Player B must either "call", by pay $2,500 to see the last card and have a chance at winning, or she must "fold" and concede the pot. Let us say Player B calls, in which case the cards are turned face up, and she sees her bad situation - she can only win if she makes the flush. However, when the last card comes up, she makes the flush and wins big!
This win, of an $11,500 pot, is bad news for our novice Player B. Learning theory: This is tremendously reinforcing, and the same reinforcement will be delivered on a variable ratio schedule if she continues to act this way in the future, even in the face of many losses. Cognitive psychology: Player B thinks her goal is to guess the top card on the deck, and thus she thinks she has done a good job by guessing correctly - the top card did indeed lead to victory. She believes she made the right move, and this belief will make her more likely to do the same thing the next time she is in a similar situation.

A professional poker player would likely have a very different reaction. This is because a professional poker player is not making decisions based on what they think the next card is. The fundamental lie of professional poker is that the next card is not determined. That is, while the next card in the deck is obviously determined --- it is what it is, sitting just over there on the table waiting to be dealt --- the professional poker player must lie to herself, and believe that it is not determined. The poker player is like the weatherman, and is not judging her accuracy based on any particular outcome.

A professional poker player would look at it this way: The pot had $9000 and Player B only had to put in $2500 to see the last card, giving her 3.6 : 1 'pot odds'. If Player B's suspicions were correct, and she could have won with a flush or a high pair, then her call would have been good, because she would then have 2.1 : 1 odds of winning (leading to an average profit of $1,200). However, a pair could not help her; only a flush would win. The odds of hitting a flush on the last card are 4.1 : 1 (leading to an average loss of $250). The call was bad. Even if she won, the call was bad. Or at least that is the lie you need to tell yourself to become a good poker player.

For the professional poker player... Learning theory: Even if the professional poker player wins the hand, the event must be punishing - it must make future events of the same type less likely, because on average the play will not be profitable. To develop successfully, a player must somehow arrange it so that she is reinforced or punished dependent upon whether her behavior was or was not the mathematically correct decision in a non-determined world. Cognitive psychology: The professional poker player does not think that her job is to predict the opponents exact hand, nor the exact top card on the deck. Instead, the professional poker player thinks it is her job to determine the probability of winning. Of course, this is usually estimated, rather than rigorously computed, but good poker players are experts at exactly that estimation. Thus, the professional poker player thinks she has done well if her opponent flips over a hand in the quality-range predicted, and if the pot odds justified the call. 

You can see this difference between the novice and professional if you watch poker on TV. It is hard to see though, because you will be distracted by the daring daring plays that make such good TV. Every poker program (with the exception of 'Poker After Dark') heavily edits the game down to just the dramatic hands. Even with all that editing, you can still observe the critical differences between the novice and the battle hardened professionals. Watch what happens when someone makes a mathematically correct call, but loses horribly when the last card is turned over. If the novice loses, they get angry. Typically, however, the pro smiles, shakes hands, and leaves. The fact of the mathematical correctness is reinforcing, even in the face of frustration over losing this particular hand. Similarly, if the novice gets lucky and wins, they are likely to think they made a brilliant move; whereas if the pro gets lucky and win, they might be disappointed by their bad play. Whether or not this particular hand is won or lost is irrelevant, what matters is making the correct moves. The only pertinent question is: If this was an imaginary world in which the next card could just as likely have been any one of the cards left in the deck, did I make the correct move.  

Wrapping up
What was the point of all this again?

The point is that 'predicting the probability of behavior' is not just some weird thing the behaviorist came up with; it is a fundamental tenant of sophisticated scientific thinking. The weatherman is not trying to predict if it will rain or not tomorrow, he is trying to predict the probability of rain. The poker player is not trying to predict if the next card will be good or not, she is trying to predict the probability of winning. Just the same, the behaviorist interested in verbal behavior is not trying to predict whether someone will say something or not in a given situation, he is trying to predict the probability. What strikes some as a 'lack of precision' or 'incompleteness' is not a defect of the science, it is a core feature that people worked hard to develop.

This is one of the big lessons that embodied cognitivists and ecological psychologists should take home from Verbal Behavior. Just as in the rest of their sciences, research into language should have, as a primary goal, determining what aspects of the world, past and present, influence the probability of a verbal response.

8 comments:

  1. Psychology
    Psychology” is the scientific study of mind and of consciousness. Psychology attempts to explain, predict, modify and ultimately improve the lives of people and the world in which they live. Psychology as the behaviourist views it is a purely objective experimental branch of natural science, which needs introspection as little as do the sciences of physics and chemistry.

    ReplyDelete
    Replies
    1. I would argue that psychology is the science of behavior.

      Delete
    2. I would also argue that psychology is "the science of behavior"... but I think we need to back that up with some heavy philosophy.

      Nolin's comment confuses some issues. Notably, the scientific process stands askew to attempts to improve people and the world. Certainly, some types of psychological research have the potential to improve people's lives, but that is no different from physics, chemistry, biology, astronomy, etc.

      Also, while I agree that psychology, even today, could use a little less introspection, most physicists and chemists that I know could use a bit more of it.

      Delete
  2. Hi Charles, I liked Skinner's book, and I agree with the proposals of your blog. In other entry, you've said that "Skinnerian behaviorism does not do a very good job at articulating the cyclical nature of perception-action processes, nor does it do the best job at evaluating what, exactly, organisms are responding to in their environments. On those topics, the ecological approach really shines." Could you explain this claim a little more? For example, is there anything that applied behavior analysis might adopt from the ecological approach in order to increase its effectiveness?

    ReplyDelete
  3. MC, that is a great request, and while I think it is a whole post's worth of effort, I will try to say something here.

    Skinner, for quite understandable practical and theoretical purposes, used the term 'stimulus' to refer to an object or a state of an object in the organism's environment (e.g. a light that is either on or off). This is good, because it makes sure we don't mistake the thing-an-organism-is-responding-to with what is happening at their sensory receptors, or, worse, what is happening in their nervous system. However, it is limited, because organisms are often responding to things best described at a higher or lower level of analysis. At the higher level there are 'situations' in Roger Barker's sense, at the lower levels there are patterns in the ambient energy or 'invariants' in James Gibson's sense. To my knowledge the only person aggressively drawing the connection between all these levels (and between all those theorists) is Harry Heft.

    Also, Skinner often disparaged perception as "merely" the first part in behaving, and hence not something he was particularly interested in. That position was also understandable, but limited. One of the central insights from Gibson's work, as well as a central tenet of Perceptual Control Theory, is that behavior is often in service of perception. (For a good summary of PCT, look here.) Skinner certainly allows for perception-action cycles in some sense, for example when describing self-stimulation (such as talking to yourself), but there is much more depth to be found in those other systems.

    Depending on what, exactly, you are trying to do with behavior analysis, it might be the case that none of this much matters. However, in other cases, I think it could make a big difference - in terms of how you set up your research/intervention and in terms of your likely hood of success. I would expect it to make the most difference in situations where "properly functioning" people are responding to non-obvious aspects of the environment, and in which "proper" behavior requires close adjustment to the state of the body-environment relationship.

    My research on perception of emotions would be a great example... if only I could get NSF or NIH to fund it :- )

    ReplyDelete
  4. Thanks for your answer, Eric. I'm not sure that your claim was fair: I think Skinnerian behaviorism has proposed useful contributions to both issues. The book Verbal Behavior described functional units of stimuli and responses at very different levels (your first issue). The concepts of "respondent seeing" and "operant seeing" were proposed to analyse the sources of stimulus control of perceptual responses (your second issue), and these concepts can be related with more recent research about sensorimotor contingencies (e.g. Noë & O´Regan). Anyway, I'd be interested in knowing more about the research programs that you've mentioned. Do you think that they might be applied to the topics of (1) verbal behavior, or (2) behavior modification in educational or clinical settings? How?

    ReplyDelete
  5. Behaviorists, including Skinner, have worked on both these issues. I have articles on exploratory behavior analyzed in the Pavlovian tradition that go way back. That said, neither are strong points for behavior analysts, and they are strong points for the ecological psychologists. Believe me, I could also go on at length about why ecological psychologists could use a better understanding of behavior analysis.

    Behaviorism and Ecological Psychology have a common historic route. Once that is appreciated, it would not be too surprising to find that they could complement each other.

    I'll see if I can write something up. I am positive these ideas would be helpful with behavior mod in at least some educational and clinical settings. A simple way of phrasing the main criticism is that it would be nice if behavior analysts routinely considered more sophisticated methods in identifying "discriminative stimuli." I don't think that is too controversial of a claim, though I can see that my earlier posts might have read as a bit harsher.

    ReplyDelete
  6. Can you give me some bibliographic references? If it can be applied to behavior modification and/or verbal behavior research, I'm be very interested.

    ReplyDelete