Specification and Perception - Fodor and Pylyshyn (1981)

After doing a mediocre job suggesting a that Gibson's needs to be defended from within the Pragmatism-lineage (as opposed to, say Descartes's lineage or Kant's), something more blunt and obvious might be in order. I have argued that much confusion was created in the past debates over ecological psychology because its critics were not treating it as part of the pragmatic lineage, and its defenders met the attack on the critics’ terms. This lead, I think, to the defenders formalizing ecological psychology in a way that loses some of the unique potential of the approach. The trouble seems to have originated, largely, in the 1981 criticism by Fodor and Pylyshyn, which was replied to in the same year by Turvey, Shaw, and Mace. While the resulting "TSM" model of ecological psychology has led to much success, I won't deny that for a minute, I think that much of the current confusion within the field of ecological psychology traces back to this exchange. Below I will go back through Fodor and Pylyshyn's paper, to point out where I think they unfairly set out their challenge, i.e., where they tried to judge ecological psychology based on premises the pragmatic tradition rejects. In the next post, I will sketch what I think the reply to Fodor and Pylyshyn should have been. In a later post, I will go through Turvey, Shaw, and Mace's paper, to show how the acceptance of Fodor and Pylyshyn's premises lead them to conclusions at the heart of current debates in the field.

Fodor, J. A. & Pylyshyn, Z. W. (1991) How direct is visual perception?: Some reflections on Gibson's "Ecological Approach". Cognition, 9, 139-196.

Summaries of Fodor and Pylyshn's position will be in blue, my comments will be black.  Bold will indicate F&P’s section titles.

Fodor and Pylyshyn, 1981

1. Introduction

The "Ecological Approach" being referred to in Fodor and Pylyshyn's title is Gibson's 1979 book... their paper is a 58 page book critique.

F&P lay out the central theme of the paper by pointing out that Gibson claims:

Visual perception is direct and requires only a selection from information present in the ambient light,

whereas,

The current Establishment theory (sometimes referred to as the "information processing" view) is that perception depends, in several respects... upon inferences. (p. 139)

F&P reference that such inferences are typically taken to require an "intrinsic connection between perception and memory" and a system of mental representations that can be transformed and used in mental computations. It should be pointed out that by the time the '79 book came out, it might be fair to pit Gibson against cognitive psychology, but Gibson was developing his approach well before the cognitive revolution occurred. Thus, while his approach was intended as an alternative to the many theories of perception that came before, it is weird to read him as primarily trying offering an alternative to “information processing” approaches.

F&P will argue for a "conciliatory reading" of Gibson, in which they attempt to assimilate many of Gibson's insights into the information processing approach - they will also forcibly deny that Gibson has achieved, or even initiated a valid alternative to the information processing approach (i.e., they are doing a "Lakatosian defense" of cognitive psychology). The will do so by claiming that Gibson's account requires constraints on its key terms which he cannot provide, without allowing in some form of inference. At the same time, they will admit Gibson's insights about a stimulus ecology, including the existence of patterns in ambient light spread over time-and-space, which organisms are sensitive to. They will admit it, that is, so long as we understand that organisms must still infer from the patterns they pick up to the objects 1) because the patterns observed are limited, and under-specify the environment, and therefore the organism must infer what the larger pattern looks like, and 2) because even with the complete pattern an inference must be made about the type of world that created that pattern.

F&P's are already backing Gibson into a corner. They have ignored two important points in Gibson's writings: 1) There is significant redundancy in the patterns that specify object properties. Thus, Gibson claims, it is not necessary to see the whole pattern for there to be specification. 2) If an organism does, at a particular time, have access to under-specifying patterns, it can always keep looking. The first point undercuts one of F&P's main arguments, but the second cuts at their foundational assumptions. Gibson is not arguing that organisms are always in unambiguous conditions, he is arguing that continued exploration can typically disambiguate the situation.  

2. The Trivialization Problem

F&P claim that Gibson's theory can be read in a trivial way, as offering a story that cannot be wrong. We might, for example, explain how people are so good at recognizing shoes by claiming that there exists a unique property of "being a shoe" which is specified by a pattern in the light, which people are sensitive to. They want you to see this (positively!) as related to Chomsky's criticism of Skinner.

This section suffers from failing to distinguish the various ways in which Gibson used the term "invariant" (to refer to object properties, certain aspects of light patterns, etc.), but more importantly it fails to understand the scientific impetus of Gibson's system. (See also the success of behavior analysis as a science, despite Chomsky’s objections.) The story F&P offer about the shoe might seem silly, but it does create a context for investigation. Scientific questions might include: Can we identify patterns that specify the configuration of properties that are "being a shoe"? and, Are people sensitive to those patterns? That this sounds weird when we are talking about shoes is due to our intuition that science might make better progress by focusing on different things, it’s sounding weird does not indicate that it is an inherently a stupid question.

F&P again assert that Gibson must account for how an organism gets from the pick-up of patterns to the perception of the object.

Oddly, F&P do not spell out this challenge. What do they mean by perception? Gibson doesn't think he needs to account for the perceived mystery, because, for Gibson, picking-up the patterns is perception. Presumably, F&P want to know how Gibson explains the construction of our internal representation of the world, but like Holt, Gibson does not think we have such a representation (or, at the least, Gibson does not think that the process of perception requires creating representations). To invoke embodied cognition: To pick up certain patterns is to become the type of organism that is more likely to do certain actions; in one step, no inference required.

2.1-2.5 The need for constraints

F&P then offer, over the course of 10 pages, several ways in which Gibson might be read as offering constraints on his claims about perception. 

1. In a section ostensibly suggesting that Gibson thinks only affordances can be perceived, F&P point out that it would cause trouble if "ecological" and "directly perceivable" were interdefined. They are correct… but Gibson does not risk this tautology! Only some ecological things are directly perceivable. More importantly, the assertion that organisms are best at perceiving objects and events on an ecological scale does not require any of the rest of Gibson’s thinking. We would not, from a generic naturalistic perspective, expect organisms to be designed to perceive objects and events on a non-ecological scale (e.g. atoms or mountain ranges). Discussion of affordances here is a red herring. 
2. F&P suggest that Gibson might claim that only properties of objects that affect ecological optics are perceived. If we are limiting the discussion to things perceived by vision, that seems fine. F&P then elaborately restate their assertion that relevant object properties, and particularly affordances, cannot be specified in light. Their major rhetorical tool here is to shift the discussion to affordances that are not likely to be specified in light, which is a dirty trick after they start by focusing the discussion on optics. Again, discussion of affordances at this point muddles the critical issue. 
3. F&P suggest that Gibson might claim that properties of the world readily available to introspection are perceived. This section is bizarre. Especially given Gibson's penchant for discussing animal examples, it is hard to imagine how he could require his system to involve introspection. F&P are just trying to erect strawmen.
4. F&P suggest that Gibson might claim that “whatever the perceptual system responds” to is what is directly perceived. This mistakes the properties of the world for the specifying energy.
5. Finally, F&P assert that perceptual error is a huge problem for Gibsonians. Here they add the assertions (not mentioned before) that perception necessarily terminates in beliefs, which can be objectively judged right or wrong. While F&P are correct that perceptual errors are a challenge for Gibsonians, it is patently odd that they discount the elaborate discussions of error in Gibson's work. It should also be clear that while perception relates to belief formation, Gibson rejects the idea that perception necessarily terminates in a dualistic belief. (See next post for alternative interpretations of “belief”)

Overall, this section of F&P's paper is very odd. They lay out a series of "Well, if Gibson thinks X, then he must be wrong", where X is not something Gibson thinks. Also, repeatedly, F&P seem to complain that Gibson's terms are so unconstrained that it would allow us to hypothesize the existence of things they assert do not exist (such as specification of a shoe). That is, they repeatedly try to force Gibson to be claiming that things exist, where Gibson is merely claiming that things might exist. There is no sense of naturalism, no sense that there is a lot that might be, but we are only trying to account for what actually is. For example, before we looked for a shoe-specifying-invariant, we might first do a study to see if people actually perceive shoe-ness accurately. Again this sounds weird when talking about shoes, but ecological psychologists routinely check to see if certain properties of the world are perceived accurately before moving on to look for specifying-invariants (e.g., do people accurately perceive the length of a wielded rod, or their ability to walk through doorways?).

3. The problem of direct detection in Establishment theories.  

In this section, F&P are honest enough to admit that the establishment view has many problems as well. They praise more of Gibson's insights, but again evade the main challenge. The most interesting part of this section is footnote 4, which argues that Gibson’s views have no epistemic implications (see next post).

4. What is picked up in (visual) perception is certain properties of the ambient light.

In this section, F&P start by providing what is essentially a Shannon-style view on "information". They then claim that such information is a matter of ontology, while specification is a matter of epistemology. Several problems are then illustrated. There are a lot of issues here - both "information" and "specify" are used in very non-Gibson ways. Some of the supposed problems are patently manufactured for the purpose of contrived attack. Two examples are particularly illustrative of the failure to consider Gibson's position: First, F&P's claim that Gibson must explain why it should be more important to the organism that structured light specifies the surrounding world, rather than it being important that the surrounding world specifies the patterns in the light. F&P's refusal to consider perception as an activity in the world, which serves further action, is astounding. Second, F&P claim that we should create experiments in which we hold the world constant, but change the patterns of light and experiments in which we change the world, but hold the patterns of light constant. But Gibson's point is that this is not possible! (Yes, it is possible if we restrict ourselves to "light", but if that is the point, then F&P are cheating again.) Any change in the world necessarily affects the structure of some energy array. Contra to F&P's claim, one cannot make something that structures the ambient energy exactly like a shoe does, without making a shoe. 

More importantly, perhaps, F&P argue that all we can ever see directly is patterns in light. This is the psychologist fallacy par excellance (see next post). What we see are objects and events in the world.

5. The "Information in the light"

Here F&P assert that "picking up information" and "directly perceiving objects" are separate epistemic acts, with the prior mediating the former. Therefore, they declare, Gibson is proffering an inferential, mediated theory. They are wrong. For Gibson, it is a single act. The rest of F&P's criticisms in this section are not really relevant, because they are wrong about the first point. F&P further assert that the organism must "know about" the relationship between the light-patterns and the objects for perception to occur. This is patently absurd, in either the Establishment or in Gibson's system. I cannot think of a meaning of "know" that would make this work. It is like saying that a person cannot swim without knowing about swimming. Recall that Gibson does not offer a correlational (as in probabilistic) notion of information. Laying out the dilemma, F&P state:

it seems plausible that recognizing X to be about Y is a matter of mentally representing X in a certain way; e.g., as a premise in an inference from X to Y. And it is, of course, precisely the notion of mental representation that Gibson wants very much to do without. We have here a glimmer of Gibson’s ultimate dilemma: the (correlational) notion of information that he allows himself simply will not serve for the identification of perception with information pickup. (p. 168)

Throughout, F&P rely on examples in which an organism, at a given moment, lacks access to sufficient specification. Repeatedly, F&P state that Gibson has offered no alternative to the process of mediation. They completely miss Gibson's point, which is that no such process is need.

6. Only properties of “effective stimuli” are directly detected

Here F&P start by claiming that the Establishment position and Gibson have much in common. In fact, they claim that a weak version of Gibson's thinking is identical to the Establishment view. The stronger version, which they admit differs from the establishment view, claims that all visible properties of the world are specified in patterns of light. While they admit this might be true in a "trivial sense", if all the light that is and ever shall be is considered, they conclude that very little specification is present in the light an organism could be exposed to in a reasonable amount of time. This section is not bad. It comes down to a disagreement about the plausibility of invariants, and disagreement about the redundancy of patterns in ambient energy. That is, it is an empirical question. The only problem is that F&P dismiss out of hand the possibility that Gibson might be right. 

F&P proceed to defend a causal model of perception, in which a string of events begin at an object and end in a percept (perceptual belief or perceptual judgement). Nowhere is there an appreciation for Gibson's discussion of the circular nature of perceptual processes, of perception as an active process, or of behavior being in service of perception. (These, by the way, are points on which ecological psychology and Perceptual Control Theory are in complete agreement).

One can tell why F&P think they are being conciliatory when they state that

On any account... percepts have causes, and among the causes of a percept will be some bounded spatio-temporal segments of the ambient optical array… Given this notion we can now ask the critical question: Is it true, in the general case, that each [segment of the array] is uniquely correlated with the structure of a corresponding layout?” (p. 172)

However, it is then claimed that, when the question is phrased this way, the answer is obviously "no".  But with this F&P dismiss off hand a most crucial part of Gibson's argument. There is no serious consideration of whether specification may exist in the "spatio-temporal segments" available to the organism. F&P then dismiss off hand Gibson's argument against lab studies. One problem is in their demand that a theory of perception account for "all percepts" as if there were a static category of things to be accounted for. But Gibson rejects this, in line with the pragmatic tradition, because he sees perception as a process. (See next post)

F&P then lay out a main argument: If, as they claim, the typical state of the organism is having access only to an "effective stimulus" that "underdetermines" the state of the surrounding world, what additional process is involved in "causing the percept"? Ignoring what is typical, organisms are often stuck with access to part of the array that is ambiguous, so the challenge is fair. However, why on earth would we think that something extra is involved in those situations? This is like asking what extra thing causes a car to go forward when one of the spark plugs is bad. There is no additional thing, there is whatever the organism does when it is, for whatever reasons, restricted from functioning in an ideal manner. There is no secret ingredient. That the organism does one thing or the other is explained by less of a process, not more of process! Gibson is articulating a larger process in which the organism takes part, F&P want to explain a process in which there is a thing (a percept) that the organism has at the endpoint.

7. What properties of the effective stimulus are directly detected?

Returning to their conciliatory mode at the end, F&P reiterate their agreement with Gibson that the traditional views have underestimated the complexity of light that the organism can respond to, and embrace something akin to Gibson's Perceptual Systems view of eye-head-neck-body-etc. systems that can be sensitive to that complexity. They also readily concede this can allow the detection change and other higher-order variables as perceptually "primitive." Finally, they concede that 

the spatial limits of the immediately detected visual properties may extend beyond the retinal field and their temporal limits may extend beyond the measured refractory period of the visual system as neuroanatomically defined. (p. 176)

The latter is, for historic reasons, quite a big concession for someone claiming to be representing the Establishment view.  F&P then proceed to give 11 pages of examples from the study of psycholinguistics and audition that they think support their points regarding what, exactly, is "transduced" when listening to language or language-like events. The section is riddled with a new layer of assumptions about how language and hearing works, many of which ecological psychologists would probably reject (see Skinner's Verbal Behavior and Sabrina's investigations of language from an ecological perspective). The examples offered typically start by a series of claims about how everything Gibson requires is "prima facie" and "ipso facto" impossible. It is worth noting that, in this section, F&P suddenly start talking about "all possible worlds". It is also worth noting that, in this section particular, F&P seem to be trying very hard to give Gibson a fair shake. In footnote 11, F&P make clear that, in their opinion, it is clearly one thing to respond selectively to equilateral triangles, and another thing to respond selectively to equiangular triangles. From an pragmatic perspective, I am not sure there is a difference (see next post). In a discussion of English vs. Pig Latin, F&P again fail to even consider the possibility of higher-order invariants that maintain structure despite transformations.

If there is a new quote in this section to ruffle our feathers, it is this one:

properties of the layout are inferred from properties of the light... so long as the organism is detecting light patterns and not layout patterns, there is no way to avoid the conclusion that perceptual knowledge of the latter is inferred; this remains true no matter how perfect the correlation between light and layout may prove to be.

For Gibson, there is a big difference between the relationship being probabilistic and it being perfect. If there is specification, we can cut out the middle-man of inference.

8. Conclusion: The problem of intentionality

Because, I guess, it is the thing to do when writing about perception, F&P end their paper with a section about intentionality. Their argument is that Gibson has an even bigger problem, because he has a theory of perception that he wants to treat as if it was a theory of cognition; but, cognition is intentional, while perception is not, so Gibson can have no traction regarding cognition. This involves introducing a series of suspect assertion, such as a clear distinction between "recognizing" and "recognizing as". For example, they claim that a primary advantage of the Establishment position is that: 

In effect, it allows us to understand seeing as in terms of seeing and mentally representing. (p. 190)

They only real response I can give here is to point out that in a dynamic systems version of perception, there is no such clear distinction. Gibson readily admits that perception develops, and that some of that development is through straightforward interaction with the relevant objects. This allows for individual differences in the perceived function of objects, but in no way requires a separate "cognitive" step to account for such differences.

Finally, F&P restart their wholesale bashing of Gibson, without repeat of their praises. They also try to hit him with examples using perceptual scenarios he has rejected and which are clearly suspect. What could F&P possibly mean, for example, by suggesting that people sometimes perceive stars as giant balls of flaming gas?!? All in all, a deflating ending.