Grapheme-Color Synesthesia as Perception without Awareness

Presenter: Nemira Gasiunas, Columbia University

Commentator: Berit Brogaard, University of Missouri Saint Louis

  • Get Berit’s paper in .pdf format
  • Get the audio in .mp3 format
  • Get the video in .m4v format or in Ogg format
  • Advertisements

19 Comments

  1. I was just about to comment on Nemira’s paper that ‘How about dorsal vision as a candidate for PWA?’, but I see that Berit makes the same point. So this comment is for Berit (a related comment for Nemira will follow):

    Berit, I am a life long supporter of perception without awareness in general and of dorsal vision as perception without awareness in particular, but I do have worries about your way of arguing for this. You make a very convincinng case for dorsal vision lacking awareness, but I suspect that most critics of dorsal vision as perception without awareness would grant that dorsal vision lacks awareness, but they would deny that it counts as perception at all. And you rely on your definiton on pp. 1-2 on this: dorsal vision counts as perception according to that definition, thus, it must be perception. But I don’t think that those who would not want dorsal vision to count as perception would accept this definition. They would say that dorsal vision may carry information about states of affairs, but it is not perception. Or, they would say (something Nemira mentions) that it is a subpersonal process, whereas perception is necessarily a personal level phenomenon. I myself reject both of these ways of thinking about dorsal perception, but I’m not sure you’ve said anything against them.

    Your definition for perception is this:

    φ is an instance of perception iff φ is a (sensory) neural process in subject S
    triggered by a sensory stimulus that (in the absence of independent
    counterevidence) disposes S to act in ways characteristic of neuro-typical individuals exposed to an external stimulus under normal viewing conditions (or
    analogous conditions for non-visual cases). (pp. 1-2 of your comments)

    A couple of questions about this definition:

    1. Wouldn’t it be enough for you to give a sufficient condition for perception in this context? I think the necessary condition brings in all kinds of potential problems, which are easily avoided if you only talk about a sufficient condition.
    2. What is meant by ‘being disposed’? Rylean disposition? In that case, your opponents will reach for the arsenal of anti-Ryle arguments.
    3. What is meant by ‘normal conditions’? Will this appeal to ‘normal conditions’ not face the familiar problems of what counts as ‘normal’ (e.g., Dretske, Fodor, Millikan).
    4. What would make ventral vision qualify as perception? Even if we acknowledge (as we should) that the ventral and the dorsal stream are communicating at multiple points, it sounds too strong to say that ventral vision ‘disposes us to act’ in any way.
    5. A related question: what counts as acting (I know you say you don’t want to engage with this ‘tedious’ question here)? More precisely, I’d include micro-saccades. Do you?
    6. Suppose I implant some really powerful artificial wiring between your retina and your motor cortex: each time certain cells in the retina fire, this will make some cells in the motor cortex fire and this in turn gets you to perform various actions. This would count as perception according to your definition, but would we really want to call this perception?
    7. Finally, some may worry that the reference to action in the definition may seem too convenient for your purposes as this will make dorsal vision qualify automatically – it may seem as if the definition were taylor-made in order to fit dorsal perception.

    Would there be a way of strengthening the sufficient condition part of your definition? Again, I’m just worried that those who want to say that dorsal vision is not perception will reject your definition altogether, so it would be good to come up with a definition that they’d find difficult to reject.

  2. Nemira, thanks for the great talk. I’d like to comment on the assumption that you use for dismissing some of the potential candidates for PWA, namely, the Dennett/Burge assumption that perception is necessarily a personal level process.

    I have two problems with this assumption, the first one is that it is very ambiguous and difficult to disambiguate (see Bermudez 2000 here). When is a mental process part of the personal level and not the sub-personal? You appeal to the concept of agency, but of course that’s just shifting the problem as agency is not exactly a crystal clear concept either. In the present context we can not appeal to consciousness to delineate personal and sub-personal (and the two distinction are supposed to be separate anyway), but then how should we delineate them? Would perception if we accept Berit’s definition count as a personal level process? More generally, if a concept of perception makes reference to two different mental sub-systems (say, a sensory and a motor one), would this count as a personal level concept? If not, I really can’t see how a personal level concept of perception can be given without resorting to ocnsciousness, which, again, in this context, is ruled out.

    I also find the general proposal that perception would be a personal level process very unappealing. It would, if I understand the proposal correctly, rule out both ventral vision and dorsal vision, as these are processes of two different visual sub-systems – the ventral and the dorsal visual sub-systems. This way of thinking about perception would also seem to automatically rule out any naturalistic understanding of perception as analyzing the constituents processes of perception would often (maybe always) involve leaving the personal level. In the case of vision, it was a major step in understanding this process when the two (maybe three) visual subsystems were discovered. If we accep the personal level concept of perception, these results are, strictly speaking, irrelevant for understanding visual perception. I would find this consequence of the personal level account very problematic – but I’m not sure how to think ofthis account in such a way that it would not lead to this consequence.

    I do take the main point of your paper to be the argument from Grapheme-Color Synesthesia and this argument does not rely on these assumptions – and this argument is both original and very convincing. But your negative arguments in the first half of your paper do – I’d be curious to see how strongly you then rely on these assumptions.

  3. I’d like to thank Berit for her interesting and incisive response, and thank you also to Bence for his comments. In many ways, what I want to say about Berit’s comments are tied up with what I have to say about Bence’s point, but I’ll try to respond to Berit’s paper in this post and address Bence’s questions in the next.

    The more I think about it, the more I’m struck by just how much of the argument over PwA centers on questions of what counts as perception and what counts as awareness. I happen to think, personally, that blindsight probably does count as an example of PwA. What I was trying to do, however, was to show that blindsight is a difficult example, an ambiguous one; one open to legitimate doubts. I think the same is true of Dorsal vision. And grapheme-color synesthesia is offered by me as a superior example of PwA, because it avoids these ambiguities. It is my view that there is no question that grapheme color synesthetes are perceiving and that this perception takes place in the absence of awareness.

    Delaying, for a moment, a discussion of Berit’s definition of perception, I think she’s right to say that “the important question with respect to blindsight is not whether it is an instance of perception but whether it is a case of unconscious perception”. The problem is that the examples she discusses which supposedly offer “exceedingly good evidence” for blindsight subjects as lacking awareness are, I think, still open to skeptical doubts. The recurring problem is, as Overgaard et al point out, with the accuracy of subjective measures of awareness. We can always ask whether the blindsight subject’s judgments about their perceptual experiences accurately represent those experiences, or whether, in denying having conscious experience, they are underestimating the extent to which they are enjoying weak visual consciousness. Nor do I think the case of GY settles this difficulty. For one thing, it is just so difficult to pin down what is being talked about when we talk about qualia (after all, almost 40 years later, philosophers are still at loggerheads over questions like whether Nagel’s ‘what it is likeness’ really picks out a genuine phenomenon!), so I have serious doubts as to whether his being ‘educated’ on the phenomenon by reading a selection of scholarly descriptions will remove the problem. And in any case, even if GY understands exactly what consciousness is supposed to be, there is still the worry that he will be unable to accurately report degrees of consciousness – he may expect that only fully conscious sensations ‘count’ and thus fail to report on weakly conscious visual experience.
    I’m not saying that this is definitely what happens; I’m just saying that it’s a worry that some people have. And it’s always going to be a problem with blindsight because the only tests we have recourse to in this area are subjective measures, usually in the form of verbal reports. But we can’t be sure that the subject’s judgments on their states of consciousness are correct ones. I think GC synesthesia avoids this problem – I’ll describe exactly why below.

    The problem with Dorsal vision is the other way round. It seems to me convincing to hold that dorsal vision occurs in the absence of awareness. But – and I can’t say it better than Bence does in his comment to Berit – I think the case for dorsal vision as perception needs to be much stronger than it is for it to count unproblematically as an example of PwA.
    The issue of what counts as perception comes up in Berit’s criticism of my own examples. She suggests that “typical cases” of G-C synesthesia (such as the pop out cases I discuss) don’t do the work I want them to: “the fact that the synesthetic color experience is triggered before the targeted grapheme is identified does not show that any unconscious processes involved in generating synesthetic experiences are perceptual processes.” I think this is quite wrong, for one very good reason. As I describe, most (though not all) G-C synesthetes have their synesthetic color experiences triggered not only by the form of the grapheme but also by the way the grapheme is classified (for example, the same form can invoke different color experiences depending on whether it is classified as an A or as an H). But for this to be the case, it must be true that the subject’s processing of the letter involves high-level cognitive processing (ie lexical processing). This makes such cases quite different from examples like unconsciously “registering colors in the retina and the LGN”. I don’t know what we could call the high-level processing of visual input that goes on in GC synesthesia if not perception; and if this contradicts Berit’s definition of perception in terms of grounding action, then so much the worse for that definition (Though I am sympathetic to the appeal to action with respect to perception, I also agree with Bence that the emphasis on action in Berit’s definition may be too strong as a necessary condition).

    So my examples of GC synesthesia count as perception of the grapheme. And I also think that they count as unconscious perception, without the need for subjective measures of awareness. If the perception of the grapheme, prior to the inducing of the synesthetic color, were conscious, then there would be no way to explain why the synesthetes showed improved detection rates over non-synesthetic subjects; all subjects would plausibly be able to identify the relevant graphemes as soon as they perceived them, and the triggered color experience would be of no extra ‘help’. So we can conclude that, in the early stages of visual processing, the perception occurs without awareness and the synesthetic color experience is what speeds up or induces awareness of the grapheme, enabling GC synesthetes to identify the graphemes more quickly. And I think this is a very neat result, because we have plausible examples (not definitions, which I’m not sure we even need, so long as we have sufficient conditions) of perception and awareness (or absence of) which don’t define the first in terms of the second, which don’t rely on subjective reports, and which best account for the fascinating test data. Which is why I think these kind of examples are stronger evidence of PwA than either blindsight or dorsal vision.

  4. Bence, many thanks for your comments. I confess to being somewhat worried by the personal/sub-personal distinction myself. Perhaps it would have been better not to use it, except that its not easy to come up with ways to mark the distinction between, on one end of the scale, instances of, for eg, proprioception (etc) where there is no controversy about information processing in the absence of awareness, and on the other end, cases of visual processing like the ones I describe, and where the really interesting disagreement occurs. In many ways, I think it doesn’t much matter, for the purposes of this debate, whether we call these first kinds of cases perception or not (of course, it matters immensely in other contexts). What matters is that there is a certain class of perceptual activity where the question whether it can occur in the absence of awareness is a truly difficult and interesting one. We could even call this class Perception*.
    I appealed to agency in my characterization of perception, and partly because I think there is something to your criticisms of both Berit’s arguments and my own, I now think I’d not want to insist on this as a necessary condition for perception, although there is clearly some sort of deep and important connection there.
    So let’s try a different tack: a distinction which increasingly appears to me to be a promising one for marking out the interesting cases is that between low-level and high-level information processing. This is hardly an unproblematic distinction, but it is one that psychologists, philosophers and neuropsychologists have all usefully employed. High-level perception is roughly the process of making sense of complex data at an abstract, conceptual level. The very fine-grainedness of low-level representation is lost, but the increased generality of higher-level processing allows disordered environmental stimuli to be organized into the mental representations that are used in cognitive processing.
    According to some theories, perception only occurs on the higher level of information processing. (Reverse Hierarchy Theory is one example of this view – see S. Hochstein and M. Ahissar (2002): View from the top: Hierarchies and reverse hierarchies in the visual system). I personally find this account compelling, but I don’t think it has to be accepted for us to be able to put the high-level/low-level distinction to good use. In other words, I’d be prepared to allow the term ‘perception’ to apply more broadly to so-called sub-personal mechanisms (for example, to the proprioceptive processes that allow us to keep our balance, as well as to visual subsystems), but claim that the interesting questions about PwA center on high-level perception. (The question how the higher and lower levels relate to one another is a further, fascinating issue).
    Blindsight, I think, if it could be shown genuinely not to involve awareness, would be one of the ‘interesting cases’, involving high-level visual representations. Can we say the same thing about dorsal vision? Unclear. But it seems absolutely clear that the cases of G-C synesthesia I discuss involve high-level representations of the graphemes, representations which are suitably abstract and conceptual.

    I apologize for the vague and tentative nature of these suggestions, but I’d be interested to hear your thoughts.

  5. Nemira, this could work, but, as you rightly say, this may not be a more solid distinction in the end than the personal/sub-personal one.

    But, luckily, you don’t really need any of these for your argument. You could just say: The data from GC Synesthesia shows us that there is PwA even according to a crazy demanding concept of perception. Let alone other, less demanding ones. So you don’t have to endorse the personal level account of perception. Would this work?

  6. Hi Bence. Thanks for your great comments. My definition of perception was meant to be a first approximation. I agree with your criticisms. The problem, as I see it, is that we cannot have any meaningful discussion of which phenomena count as PwA without a definition of “perception”. I think the same point applies to almost any discussion in the philosophy of perception. For example, how can we answer the question of whether there are high-level properties in the content of perception if we don’t know what perception is? We can’t. For these debates to make any sense, we need to figure out what perception is.

    Having said that, let’s look at my proposal as a sufficient condition:

    φ is an instance of perception if φ is a (sensory) neural process in subject S triggered by a sensory stimulus that (in the absence of independent
    counterevidence) disposes S to act in ways characteristic of neuro-typical individuals exposed to an external stimulus under normal viewing conditions (or analogous conditions for non-visual cases).

    It is very difficult to say anything meaningful about “normal viewing conditions”. But I will try. “Normal viewing conditions” refer to a range of different conditions that do not involve atypical lighting conditions, great distance, etc. So, noon and sunshine is just one kind of normal viewing condition. 5 pm and cloudy is another. Whether a condition is typical or not will depend on the object in question. Forexample, for an object that is almost always spinning, the object must be spinning relative to the perceiver for the viewing condition to be normal.

    This reminds me of a discussion some Australian scientists had about the coral reef in North-Eastern Australia a couple of years ago. They were discussing the true colors of the reef. If I remember correctly, the problem was that some rocks had different colors under water and above water.

    In this case, I would say that the normal viewing conditions for viewing these rocks are under-water conditions. Of course, there will be a lot of borderline cases of normal conditions. But I find that unproblematic.

    As for “disposes to act”: I don’t have any problem with behaviorism when restricted to cases like perception. If we don’t give a functionalist or behaviorist account of perception, then what options are left? An anatomical definition won’t do, as V1 (for example) is activated in memory, visual imaging, etc.

    As for the notion of action, saccadic eye movements may be part of the relevant range of actions or behaviors. But it is not the only form of relevant behavior. The reason for this is simple. Saccadic eye movements can be found in cases of non-perception (e.g., visual imagery, in the day dreaming sense). What I had in mind was cases of ordinary behavior. Sitting down on a chair you are seeing, picking up a fork, turning toward a person addressing you, jumping in your seat when you hear a loud noise, etc.

  7. Hi Nemira, Thanks for your great comments. What is your definition of perception? I don’t see how one can argue that a particular phenomenon is a case of perception without a definition of “perception.” I don’t think the unconscious processes underlying color-grapheme synesthesia are very good cases of perception.

    As for dorsal-stream representations, they seem to me to be obvious cases of perception. But yes, people could dismiss my proposed characterization of perception. But before we dismiss dorsal-stream representations as cases of perception, I would like to see how perception is characterized if not the way I did it.

    As for blindsight, you can question whether blindsighters really have no visual experiences in their blind field. But this doesn’t help your case, because there is little doubt that blindsighters have no color experience. So, that makes color processing in blindsight an excellent case of unconscious perception.

  8. Thanks, Berit, for these clarifications. I think what you propose is much more plausible as a suffficient than a necessary and sufficient condition, but, as we agree, you only need the sufficient one for the present purposes.

    I think that something like what you suggest must be right, but I still have a couple of worries about your specific proposal:

    a. how about my example (6 above)? It would qualify as perception according to your definition. Would you bite the bullet?

    b. Also, although I think that the definition of perceptions must be somewhere along the lines you propose, I am worried that your opponents will say that you arrived at this sufficient condition by just conveniently adding the features of a case you want to count as perception (dorsal vision in this case) to some generally accepted features of perception that everyone agrees on. In this sense, you give a definition that automatically makes dorsal vision qualify – but then those who do not want dorsal vision to qualify will not agree with your definition.

    c. Finally, the definition you propose reminds me of Quine’s way of thinking about perception (his concept of perceptual similarity in the first chapter of his Quine 1974 and later). If you accept the similarity, what do you say about what I take to be the standard criticism of Quine on perception, i.e., Churchland 1976?

  9. Hi everyone, really interesting discussion!

    I agree that we need a definition of perception if we are to get clear on this debate. I am of the mind that a perception is something like a sensation with a thought. To use some standard terminology we can postulate a family of mental properties that serve to track perceptible properties and which preserve the similarities and differences in those external perceptible properties. We can call these mental properties ‘starred properties’. So, we keep track of physical red with red* which is a mental property that differs and resembles green*, orange*, etc in ways that preserve the similarities and differences between red, green and orange. But to perceive is to also have a perceptual thought, something like ‘there is red there’. So to have a perception on this view is to have a red* sensation and a perceptual thought. Is this the kind of complex that Brit calls a ‘(sensory) neural process’? Or is she only trying to capture the sensation bit?

    Also, I wonder what Nemira or Brit think about cases of subliminal perception? Especially in masked priming paradigms there seems to be clear cases of perception without awareness.

    finally, I wonder whether we are assuming that if there is perception without awareness that means that there is phenomenology without awareness?

  10. I thought the point of synaesthesia is to rebut what we may call following Travis the silence of the senses, the idea that sensations are epiphenomena, at most accompany perceptual judgments, or they do not exist. Why is it a case of perception without awareness if the synaesthetic is conscious both of color and say the shape of a digit or a number? What am I missing?

  11. Hi Nemira —Thanks for a really interesting and thought-provoking paper. I want to take up quite a different line of questioning than Bence and Berit. I have no problems with the notion of unconscious perceptions or unconscious colour perception, so I am not in need of convincing. But I did want to ask about the explanatory adequacy of your hypothesis qua an explanation of the experimental results. (Perhaps this is what Aspasia is asking.)

    In your response to Berit and Bence (hereafter, B & B), you say:

    ‘If the perception of the grapheme, prior to the inducing of the synesthetic color, were conscious, then there would be no way to explain why the synesthetes showed improved detection rates over non-synaesthetic subjects; all subjects would plausibly be able to identify the relevant graphemes as soon as they perceived them, and the triggered color experience would be of no extra ‘help’. So we can conclude that, in the early stages of visual processing, the perception occurs without awareness and the synesthetic color experience is what speeds up or induces awareness of the grapheme, enabling GC synesthetes to identify the graphemes more quickly.’

    Right. So you’ve identified the lovely chicken-and-egg problem that arises for any experiment that seems to show synaesthesia as helping (or hindering) normal perceptual processing — e.g. It makes the subject faster at recognizing the 2 among the 5’s. As you have said, the standard way of classifying various forms of synaesthesia is in terms of inducers and concurrents—in G-C syn, the perception of the grapheme ‘triggers’ a syn. colour. But if this is what happens, it is difficult to explain how the syn colour could help with the perception of a grapheme. After all, in order to see the colour, you have to have identified the grapheme, so how could it be that seeing the colour helps with grapheme perception? To put this in more general terms, if the inducer must be recognized in order to trigger the concurrent, how could the concurrent ‘help’ to recognize the inducer? Surely the inducer must already have been perceived? But if it had already been perceived, then the concurrent could hardly be said to trigger it. And around you go.

    I take it that your response to this problem is to posit an unconscious ‘loop’ in the process. And below I contrast that with an explanation in terms of genuine pop-out, just to see how the two differ.

    A. Unconscious Shape: The subject saccades about, foveating on each letter. When the subject foveates on the 2, the shape of 2 is encoded unconsciously and then via a higher level process, this shape is identified as a particular grapheme, a 2. This recognition of the 2 then triggers the conscious colour of 2’s for this particular syn subject, say pink. And then the subject perceives the H, consciously, as pink.

    B. Pre-Attentive Pop Out of Colour : As the subject scans the stimulus array, a conscious perception of the target’s synaesthetic colour is triggered, say (conscious) PINK. This representation of PINK captures the attention of the subject, who then saccades over to its location, that of the hidden 2. Now the brain can recognize the shape of the 2 and identify it as a 2, which is then seen, consciously, as a pink 2.

    I take it that Ramachandram would go for the second explanation, which is an explanation in terms of genuine pop-out (which does not requires serial search). However, it is still not clear how, on this explanation, the PINK representation is triggered without the shape. After all, the stimulus does not contain colour so you cannot give the standard explanation of pop-out. On the standard explanation, low level processes catch the colour difference between the target and the background, and this is what directs attention towards the target. Serial search is by-passed and that is what makes the process so fast. But presumably that can’t happen here given that there is no colour present. Another empirical problem with this sort of explanation is simply that many G-C syns do not experience pop-out even though they are still faster at the task. So it looks as if the syn’s attention is not drawn to the target by the colour, although they may see the colour after they have recognized the target. This recent result is from Jamie Ward’s group (2010), “Grapheme-colour synaesthesia improves detection of embedded shapes, but without pre-attentive ‘pop-out’ of synaesthetic colour” And this view seems to be carrying the day — i.e. attention to shape is needed for syn colours to be experienced.

    That takes us back to the first option, A. Here, the first part of the process is unconscious: the subject serially fixates on each grapheme and eliminates them one by one, until finally, the subject fixates on the 2. Attention reveals shape, the shape is identified as a particular grapheme. All of this is unconscious. However, now the syn colour is triggered. Then the subject sees, consciously, the 2 as PINK. (Perhaps this seems to the subject to be pop-out simply because PINK is conscious first and so it seems as if pop-out has occurred even though it hasn’t.)

    Still, Option B doesn’t seem to explain is why syns have a faster response time. After all, on this story, you still have to see the shape and identify the grapheme PRIOR to triggering the syn. colour. So presumably the non-syn, who does not have colours associated with graphemes, simply identifies the grapheme. But then why would the syn be faster than the non-syn subject? And what role does the perception of the colour play in identifying the grapheme?

    I suspect that neither of these two options solves the chicken-and-egg problem. Option A provides an answer as to why the syn is faster, but it doesn’t explain how the syn sees the colour in the first place. Option B explains how the syn colour is triggered, but it doesn’t explain why the syn is faster. My guess is that, as long as the shape processes and colour processes are treated as a whole, as not providing any useful information unless there is an answer about shape or answer about colour, the problem won’t get resolved. That is, you have to talk about the parts of both processes and how they interact.

  12. Thanks to everyone for these fantastic and very helpful comments. In this first post I’ll focus on the points raised by Berit, Bence and Richard.
    Bence, you wrote: “You could just say: The data from GC Synesthesia shows us that there is PwA even according to a crazy demanding concept of perception.” Berit’s comments show this not to be the case – she thinks that the unconscious processes underlying color-grapheme synesthesia are “not very good cases”. No doubt she is not alone. So clearly there needs to be at least some argumentation to show that these processes should be classified as perceptual.
    Do we need a definition, as both Berit and Richard seem to think? I really don’t know. For sure, it would appear to be desirable – in all sorts of contexts, and not just the unconscious perception debate – to have one, but it’s not obvious to me that people writing on this are anywhere close to a consensus on a list of necessary and sufficient conditions. I have deliberately avoided trying to give one, in any case. I used phrases like “a test for perception” (and for “awareness”) since my purpose was simply to show that G-C synesthesia counts as an (excellent) example of PwA.
    One thing I can usefully do is to say what I think is wrong about various purported definitions of perception. As is obvious from my paper, I think it is wrong to define perception in terms that make awareness of the percept a necessary condition. I wonder whether Richard’s suggestions about perception (of, say, red) as sensation of red with a thought ‘there is red there’ might come close to such a definition. First, because the term ‘sensations’ is often taken to presuppose their being conscious (shorthand, as it were, for ‘conscious sensations’). Second, because, whatever we take a thought to be, the expression “the thought ‘there is red there’ seems difficult to make sense of if we’re not assuming that the subject is consciously having this thought. Richard, since you seem to be sympathetic to PwA in subliminal perception, so perhaps you could say more about how such thoughts, as well as their accompanying sensations, are supposed to be unconscious. (I will comment in a separate post about what I think of masked priming cases and their ilk).
    Now to the question of the connexion between perception and disposition to act. When I wrote the piece I was sympathetic to the action-guiding criterion. Now I hesitate. I never wanted to hold that it was a necessary condition but the points Bence raises, especially the thought-experiment (his point 6), make me doubt whether it is even a sufficient one.
    But even if you do want to stick to your guns about the ‘disposition to act’ definition, I’m not sure that G-C synesthesia obviously fails by your standards. The synesthetic experience, as I’ve emphasized, speeds up the subjects’ ability to pick out a given letter, or shape, or what have you. ‘Picking out’ is a form of action. So unconscious perception of the grapheme influences a subject’s actions. Isn’t that something like what you were asking for?
    I’d be interested to know more about why, Berit, you think the G-C example is not a good one, especially in the light of my comments (vague as they were) about high-level and low-level processing. I wouldn’t want to claim that high-level processing of the sort I mentioned in my reply to Bence count as necessary conditions (surely there is an absence of high-level processing in most pain perception), but I think that the conceptual elements involved in high-level processing – which it seems to me that we must assume to be taking place in the G-C syn examples, for the reasons I mentioned- are very plausibly sufficient conditions. If you want to deny that we can apply the term perception to levels of information processing at which linguistic concepts are being employed, then, first, I’m not sure were talking about the same thing when we talk about perception, and second, I’d like to know what you would call them?

  13. Hi Nemira,

    I am definitely not taking sensations or thoughts to be necessarily conscious. I think that when we say a thought is conscious or not is just that we are aware of that thought or not. The same for sensations. So an unconscious thought is one that I am not aware of myself as thinking. An unconscious perception would be a sensation and thought both of which I am unaware of being in. So, I don’t think the model I suggested builds awareness into the definition of perception.

  14. Hi Apasia, thanks for your comment. The perception without awareness in the G-C synesthesia cases is supposed to occur within a very narrow time-frame, namely the time between which information about the grapheme is taken in, triggering the concurrent, and the time the grapheme is consciously identified and thereby, say in pop-out cases, picked out from amongst other, similarly shaped graphemes. This timeframe is typically fractions of a second (see, for example, Laeng, Svartdal & Oelmann (2004) for data). Nonetheless, it is the unconscious perception during this tiny timeframe that is required to explain how the G-C subject does subsequently become conscious, and much more quickly than controls, of the letters they are required to pick out.

  15. To Aspasia: in addition to my previous comment, as a clarificatory point: consciousness is involved in the experience of the concurrent (the subject has a conscious experience of, say, redness). However, until the conscious experience of redness leads to the identification of the letter R, say, I would not want to say that it was in any way conscious perception of the letter R.

  16. Thanks, Kathleen, for a truly illuminating analysis of the possibilities and problems that emerge when we consider on a deeper level pop-out cases of the kind I discuss. Thanks also for directing me to the Ward paper, which I hadn’t read, and which is very thought-provoking.
    A number of points bear discussion:

    On the basis of Ward et al’s findings, I agree that your (B) looks, at least initially, to be not the correct account; since most subjects do not report seeing pop-out, it seems implausible to suppose that some sort of synesthetic pop-out is what speeds identification of the grapheme. But there are some puzzling questions we can ask about Ward’s claims about synesthetes and pop-out. As you mention, he tells us that most of the synesthetes reported, in any given test, dramatically reduced or completely absent color experiences corresponding to the graphemes. But he also notes that there is a small number who reported seeing 50% or more of the graphemes in a given test as colored, and also that these subjects were significantly better at identifying shapes than not only the non-syns but also the other syns (the means for these three groups are 62, 34.3 and 41.3 per cent correct, respectively). If the thesis about attention is correct, how are we to explain this difference? Are we to treat the synesthetes who do experience pop-out as a completely different case from those who don’t? Are we to posit different mechanisms working in the two cases – a process more like (B) in the case of the 50 percenters and a process more like A in the under-50 percenters? But then maybe it is not synesthesia as such that improves RT for subjects, but varieties of synesthesia, and in different ways. This does not seem like a satisfying answer, since I think we’d like to explain what’s going on in synesthetic subjects in a theoretically unified account. Then again, the correct explanation doesn’t always correspond to what we want it to be! Of course, there are as you point out other good reasons, separate from the empirical data, for why we should doubt that an account like (B) is the right one, even in the case of the 50 percenters, namely the fact that it doesn’t explain how the color representation is triggered without the subject already having perceived the shape. Ill return to this point below.

    In any case, we still have to explain what’s going on in the cases where phenomenal color experiences are not reported by synesthetes. One move we might be tempted to make in order to bridge the unpalatable gap, between the 50 percenters and the under-50 percenters is to suggest that the subjects who claim to detect less than 50% of graphemes as colored (typically around 30%) have color experiences of more graphemes than they report, but forget that they had such experiences (a similar sort of explanation has been offered for masked priming cases). After all, we are only relying on subjective (linguistic) reports of awareness on the parts of the subjects, which are, as I’ve been at pains to point out, problematic as measures of awareness (of course, it’s not clear whether there is any other kind of measure we could possibly apply in this sort of case). This sort of hypothesis is supported by Ward’s claim that synesthetes reported the colors appearing piecemeal: ‘I definitely do NOT see all the colors in one go. I have to attend to the symbols/shapes or process them in some way, and then it has a color attributed to it.’ If subject experience colors sequentially, there is more chance that they will forget the initial ones and remember the later ones. So perhaps these subjects enjoy improved reportability on the basis of a large number of color experiences, but forget that they had those color experiences that they experienced earlier.

    But though this is plausible in the case of partial reports of phenomenal concurrent experiences (of any degree), it doesn’t seem plausible in the cases where there are no experiences of synaesthetic color in any trial. The idea that one would forget having color experiences for a certain number of specific graphemes (whilst remembering ones experienced later) seems, in the light of the number of graphemes and the serial search hypothesis, explicable. The idea that one would forget having any color experiences whatsoever seems altogether less likely. And since Ward claims that these cases form the majority, we need some other way to explain these cases.

    I think these sorts of completely non-phenomenal cases are the ones you have in mind when you pose the chicken and egg problem. And you’ve had me scratching my head for several days trying to work out a way that we can explain these latter sorts of cases; that is, how subjects can consistently report an absence of phenomenal experience whilst also showing consistent improved RTs. But I wonder whether there might be a position which bridges the gap between A and B (let’s not call it (A 1/2), though I’d like to)? Some of the motivation for integrating a B-like account into our explanation comes from recognizing that a) 50 percenters exist, and their phenomenal experience seems to significantly improve their RT, and b) There may be issues with the subjective reports of the syns who report phenomenal experiences that correspond to less than 50% of the graphemes but more than none.
    So here’s the thought (and forgive me for any potential idiocies because i don’t know nearly as much about selection tasks in normal perceivers as I should):
    There is an initial stage of data processing – in which I take the subject’s “foveating about” and the subject’s “scanning the array” to be roughly similar processes. Attention is paid to each letter that is scanned but the attention is fleeting and the information received about it is unconscious, though high-level (i.e. encoding information about lexical concepts). For normal perceivers, after this initial period of fleeting attention, more ‘earnest’ search is engaged in, where the letters that were previously fleetingly scanned become more carefully attended to. So we have two (or more?) levels of attention. The advantage for the synesthete is that the initial, ‘fleeting’ attending to the grapheme may produce the conscious experience of the concurrent. The conscious color experience helps the subject to focus the ‘better quality’ attention on the grapheme in question, and this enables them to consciously pick it out the grapheme or the embedded shape. But the picking out process is separate from the initial identification of the grapheme, which takes place during the early stages of fleeting attention, as the subject is saccading about during the serial search. And this early stage is similar for synesthetes and non-syns alike.

    This account enables us to make a link between on the explanatory power of, on the one hand, (A) for how the subject sees the concurrent color, and on the other hand (B) as to why the syn has a faster RT. I realize, however, that its plausibilty depends on there being some color experience on the part of the synesthetes, something which Ward claims many of them deny having. I have raised questions about that claim but more work would need to be done there. I think there are also more questions to ask about the projector vs non-projector distinction but that will have to wait for another day!

    I’d be very interested to know what you think, and also to learn more about what you said about keeping shape and color separate. (We’re almost at the end of the conference – sorry for the delay in replying to you – but perhaps if you don’t get chance to write before friday, you could email me? My address is nhg2105@columbia.edu). And thanks once again for getting me to think so carefully about this!

  17. Hi Nemira,

    Yes, it’s a very interesting article isn’t it? Not the least because it raises as many questions as it resolves.

    For the purposes of our discussion, we should probably distinguish two different kinds of pop-out problems, single grapheme or ‘odd man out’ pop-out and embedded figure pop-out that I (at least) have been running together. If you have a odd man out pop-out, then there is one grapheme that pops-out as a result of its colour, say the single 2 hidden among an entire field of 5’s. The second kind of pop-out is the one investigated by Ward et al.: here, a group of ‘hidden’ 2’s together form a shape, a triangle or square, say. And it’s the composite shape that is supposed to pop-out.

    The chicken-and-egg problem applies most clearly to the single figure experiment. In the normal perceptual case, in which a 2 really does have a different (stimulus) colour than the 5, pop-out is supposed to occur because the colour of the 2 is pre-attentively recognized as different. It draws the eye towards it, thus side-stepping the need for serial search for the 2. But it is hard to see how explanation will work for the synaesthetic. If you have to recognize the 2 as a 2 in order to ‘call up’ the right synaesthetic colour (conscious or unconscious), then you would have already solved the problem of finding that target 2 among the 5’s. How could the colour help to identify the grapheme if the colour sensation (conscious or unconscious) occurred after you identified the grapheme?

    The problem of how syn colour helps in an embedded figure task could presumably have a different explanation (because it involves two layers of shape recognition, the initial graphemes and the composite shape)—and thereby more easily avoids the chicken-and-egg problem. Just so that everyone is in loop for this discussion, here is what Ward et al. propose (after first eliminating pop-out of the embedded figure as the explanation):

    “There is no convincing evidence from our study of ‘pop-out’ in grapheme-colour synaesthesia. The percen- tage of correct trials is far lower than one would expect if colours were perceived across the array (e.g. based on data from Hubbard et al. 2005). Moreover, the majority of synaesthetes do not report colours across the entire array and, for those who do, the colours are noted to emerge as their focus of attention shifts around the array. This is perhaps not surprising given that synaesthetic colours depend (to some degree) on perception of the associated grapheme, unlike regular visual search paradigms in which colour perception is independent of grapheme processing. Nevertheless synaesthetes do outperform controls on this task, so what might explain this, if not pop-out? If synaesthetic colours emerge within a window of attention but are not necessarily restricted to single graphemes then this could allow localized grouping within that region (based on the Gestalt similarity principle). Just seeing two red graphemes in a vertical or slanted line would provide important clues as to the shape’s identity. This may explain why some synaethetes who do well on the embedded shapes test (e.g. cases CHP and AAD in Hubbard et al. 2005) do not necessarily do well on regular visual search (reported as CP and AD in Sagiv et al. 2006). This kind of mechanism may also facilitate the search process (where to look next) by directing attention to a specific location or preventing return of fixation to the inspected area.” (p. 5)

    Okay, so on the Ward view, ‘attention’ can take in more than one grapheme. If those graphemes are seen as having a single colour, then perhaps (something like?) a low resolution spatial filter for colour could give you a partial shape or shape feature, one that is unique to one type of the embedded figures (e.g. only the square has a 90 degree corner and the 2’s form a 90 degree corner). If so, the synaesthete would see the square without having to look at all of the rest of 2’s in the composite figure (or would know in what direction to look to confirm or disconfirm the hypothesis that the 2s form a square.)

    It’s not clear to me here why you would need synaesthetic colour to do that sort of shape processing. That is, if you have identified a few 2’s and their relative positions, then you could surely process them for shape (they are now mentally tagged as being the targets). Perhaps Ward et al. have something else in mind, or think that the colour processor is a lower level or faster mechanism. In any event, if you take the explanation as given, it is an explanation that manages to side-step the chicken-and-egg problem. But it still leaves us no explanation for what happens in the ‘odd man out’ case, where the synaesthetic subject is searching for the single 2 and can do this faster than the normal subject.

    The sort of alternative explanation I had in mind in my last post applied to the ‘odd man out’ paradigm, an explanation that does not posit unconscious perceptions of colours or shapes. This is not a novel solution for a chicken-and-egg problem, at least of the evolutionary sort: just think about a time when there were no chickens and no eggs but rather chicken and egg precursors. Now apply this to processing time as opposed to evolutionary time.

    This is sort of a long story, but I will come around to the punchline after a few circles. One of the experiments that we recently did asked about the relation between synaesthetic colours and grapheme shapes. As you know, every synaesthete has his or her own individual colour for each grapheme. But what if you were to ask the question (about the relation between shape and colour) as a question about second order correlations? Here we asked: if you looked at the colour-shape pairings for an entire group of grapheme-colour synaesthetes, would you find any relation between the type of shape and the type of colour in each synaesthete? A number of different research groups have asked this question simultaneously over the last 12 months, so there is now good evidence that there is a correlation between similar grapheme shapes and similar syn colours. Thus ‘d’ ‘p’ and ‘b’ are often all one colour, for example blue. Brang and Ramachandran have a paper coming out in Neuropsychologia; Watson, Akins and Enns have a similar paper under consideration. Our group found a more specific correlation however: there is a second order correlation between HUE (not colour) and shape. That is, if you look at the colours of, say, b, p, and d, you will find that those graphemes are very close together in hue for the majority of synaesthetes (although a different hue for each individual)—‘b’ may be a royal blue and ‘d’ a more turquoise blue and so on. But whether those letters are light or dark blues is random. Thus it is the blueness that is correlated with shape, not the luminance of the syn. colour. (We don’t know about saturation yet as at the time, that data was not available.)

    That seems like a pretty odd result until you think about the problem of reading (which after all is where letter recognition normally occurs). On the normal printed page, the stimulus contains BLACK letters against a WHITE background—so every grapheme, regardless of its identity, has a high luminance contrast with its background. When you are reading, then, your eyes are moving between shapes defined by continuous luminance contrast. Suppose you have the letter string ‘s p a X e’, where ‘X’ is a letter to which you have not yet attended. You want to determine what letter is at the X location. That’s easy: you guess ‘c’ and at the same time expect to see the shape of a ‘c’ as you saccade to the spot. If you are a synaesthete, you also expect to see something that triggers the special c colour—light blue. What you get from the stimulus, then, when you saccade to that X place, will be at odds with your own synaesthetic representations of the letter, which is both c-shaped and LIGHT blue. If a ‘cue’ is wrong as many times as it is right, it counts as ‘pure noise’. And hence, luminance is noise vis-à-vis the visual search in reading for a synaesthete.

    Okay, so the take-home message here is that these experiments suggest that grapheme shape and synaesthetic colour are tied together in vaguely law-like ways. But what it also suggests is that the kind of properties that give (or might give) the synaesthete an advantage in a search task need not be the full-on representation of synaesthetic colour OR a full-on representation of the shape of graphemes. In the example, above, of trying to figure out the middle letter of ‘s p a X e’, the explanation goes like this. You foveate on the target spot; what is encoded, immediately, has a content something like “open-round-shape high-B/Y-ish chroma” and that limits the possibilities to any graphemes with an open ‘c’ shape and those that are more B than Y: A light blue ‘c’ fits these parameters, hence your hypothesis is confirmed. Now you see the c as light blue. All without processing the full shape representation of the ‘c’ or identifying it as a ‘c’.

    So one factor in why the synaesthete is faster on these tasks could be tied to how we represent shape and colour: if they have multiple dimensions, these dimensions might be encoded separately. If they are encoded separately, and the relations between shape and colour are second order relations between dimensions (not the full-on representations), then, in effect, you’ve gotten rid of the chicken and egg: you’ve broken down shape and colour so that their ‘precursors’ are used and use-able prior to perceiving shape as a whole or colour as whole. There are no unconscious representations of colour or shape, rather there are partial representations of shape and ‘colour’ precursors.

    The same sort of explanation could be given for the ‘odd man out’ experiments. You don’t need to recognize the shape of the 2 per se. Rather, some perceived shape feature triggers the conjoined hue representation, and that hue representation confirms that you have found the 2, not one of the many 5’s. Here, this ‘short cut’ would work only if the 2 and 5 have a different (syn) hue for a given synaesthete (which was the case for the Ward experiment).

    I realize this is an odd way to think of visual processing, not in terms of colour or shape representations but in terms of shape precursors and ‘chromatic’ properties. But the view is motivated by general considerations about human chromatic processing—and then came the dawning realization that we might be able to explain some of he puzzles about synaesthesia. Which is what has been motivating our experiments.

    As you said Nemira, there may be no unified explanation of why some synaesthetes have perceptual advantages/disadvantages while others do not. You point to one good reason in your paper, namely that higher level cognitive processes seem to be involved in grapheme-colour synaesthesia. (If a synaesthete interprets the identity of a grapheme based upon its context within a word, this suggests the involvement of some reasonably high level processes.) But if you go that route, then synaesthesia might alter visual processing at multiple levels, and no always in exactly the same ways, depending upon how the synaesthesia developed or was acquired. And most complex phenomena will come in a few varieties. So I am not discouraged by this lack of unity, rather heartened by the clear complexity of what seems to be both a cognitive and perceptual phenomenon.

  18. @Richard. Thanks for your comments! Here is a quick note in response.

    “I am of the mind that a perception is something like a sensation with a thought.”

    Doesn’t that rule out perception without awareness? Can you have sensations you are not aware of?

    Unconscious perceptions (aka perceptions without awareness) do not have a phenomenology. You can’t have phenomenology without awareness.

  19. Hi Richard,
    I see from your further comments above that you don’t treat sensations as necessarily conscious. I reply to your other comments below.

    Hi Bence,
    I suppose I should address your counterexample to my characterization of perception before the conference ends. Sorry, it has taken me so long to get back to you. Your counterexample was this:

    “Suppose I implant some really powerful artificial wiring between your retina and your motor cortex: each time certain cells in the retina fire, this will make some cells in the motor cortex fire and this in turn gets you to perform various actions. This would count as perception according to your definition, but would we really want to call this perception?”

    I think I am happy to call this a case of perception. I don’t feel I am biting a very big bullet by saying that. In blindsight, the information that govern the blindsighter’s actions in cases of color perception is mostly information from the retina and LGN.

    The example seems to pose a counterexample, I believe, because single cell activations don’t normally trigger actions in us. However, if – in accordance with your scenario – I stick my tongue out every time I see a cat, then the process clearly is a case of perception (vision for action) (I am here assuming that the little triggers of the cells in the retina add up to me sticking my tongue out).

    Similarly, if one of my cones is hit by red light, and I do a happy dance, then we have a case of perception (vision for action).

    I suppose that I don’t have very strong feelings about what does and does not count as perception once we leave the realm of conscious perception. But we do nonetheless need a definition of perception. Otherwise, most debates in the philosophy of perception become meaningless (e.g., “Are there higher-level properties in the content of perception?” — well, the answer to that depends on what we mean by “Perception”).

    As for Quine’s perceptual similarity, I thought this was a hypothesis about the similarity of stimuli. Two instances of red light hitting my retina are similar if I respond in similar ways to them. I would say it’s a different notion, even though I see the similarity. I don’t have a good reply to Patricia Churchland. How does her objection affect my characterization of “Perception”?

    Hi again Richard,
    Subliminal perception may be a case of weakly conscious perception. It is hard to demonstrate that there is no awareness. If there really is no awareness, then — yes — it’s a good case of unconscious perception, as far as I am concerned.

    I am not sure whether a sensation, as you construe the term, is the same as what I call a “sensory neural process.” I just meant a neural process that occurs at least partially in the relevant sensory module. For example, a visual sensory process must take place, at least partially, in the visual cortex.

Comments are closed.