JAMES CARNEY AND MARÍA ANGÉLICA MADERO

Thinking about images





Photographs, digital pictures, telescopes, security cameras, images produced by machine vision, media, cinema, series, scientific images, radars, technological images, sketches, technical images, manuals, paintings, drawings, murals, statues, data, stains, dust, infographics, reproductions, reflections, camouflage, mental images, ideology, unconscious stream, imprints, imaginary images, hallucinations, apparitions, dreams, afterimages, visions, icons, symbols, fantasies... These we find under the name of images.

This text will provide a vocabulary for talking and understanding the visual world.
Santiago Pinyol, Iphone ♾ and IPhone ♾ Pro, 2021



Why do we need to think about images?



The Book of Kells, 9th Century, Dublin

There are now more images in the world than we can usefully count. It wasn’t always like this. In the past people might have had to make a pilgrimage to see an image: they walked through mountains and towns to get to the place where images were kept. Nowadays, the number of images you are able to see depends on the speed your thumb moves. YouTube viewers watch over one billion hours of videos on its platform every single day. And on Instagram, each day sees more than 300 million new photos uploaded. In one minute alone, there are 510,000 comments posted and 293,000 statuses updated.


Laura Emsley, Installation views, Modern Minds, Maison Daura, Saint-Cirq Lpopie, 2014

The increased rate of image production tracks the productive power of human cultural technologies. These began with the dyes and engraving tools of the Palaeolithic, evolved through the printing press of the early modern era, and take important expression in present-day methods of digital production. This technological expansion means that human culture is predominantly image based, and that no appreciation of this culture is complete without a reflection on images.


Moreover, we do not just see images in the external world; we also think in images. Sigmund Freud declared that our unconscious is built in images, rather than words. Imagination? Eye effect? Visual perception? Saccadic ocular movement? All of these deal with the transmission of visual data between what is outside the skull and what is inside it. We encounter internal images in dreams, hallucinations, and at the borders of sleeping and waking.



In what ways can we classify images?


Researchers in visual studies have proposed many ways to classify images. One such schema is W.J.T. Mitchell’s genealogy of images, which classifies images into 5 categories:

1. Graphic: Pictures, statues, designs, paintings
2. Optical: Projections, mirrors
3. Perceptual: Images collected by the senses
4. Mental: Dreams, imagination, creativity, ideas, memory, fantasmata
5. Verbal: Metaphors, descriptions, writing etc.


Yayoi Kusama, "Aftermath of Obliteration of Eternity," 2009.


This is a fruitful conceptualization of the world of images because it outlines the conflict under the name of “image”. But it is important to realize that models only approximate reality; there is no such thing as the perfect model of the image.

Mitchell’s classification shows that there are mental images as well as visual images. Image comes from imago which means imitation and imagination. An image therefore has an appearance; it is seen. But an image can also be a mental construction, an imagination. Therefore an image is not solely a representation of something. The image has a reality in itself. In other words, images are not a mirroring of the world (an imitation of it) but images create the world, they produce meaning.

This is why pictures can be destroyed, but not images. Images don’t disappear. Its material presence might not be here anymore, but it can survive in narrative. Take, for instance, representations of the Prophet Mahommed: these are images that wield power in virtue of not being visible. This is an example of an image surviving in a narrative when the image itself is absent.

Another example is in relation to resolution. When the resolution of visual images was coarser, there was more space for imagination. Viewers were able to interpolate into the image. But when the definition of the images grew, people needed less cognitive and perceptual effort to engage with the image. High definition images direct imagination and therefore leave less room for us to freight the image with our desires and expectations.


Can we define what image is?


Though no definition can satisfy all images, definitions can be helpful in clarifying our intuitions about images. One possible definition holds that an image is a significant surface that connects two or several points; the structure of the image is the assemblage of information and/or stories (different elements that compose it). But rather than centering on the definition, we can centre in the meaning that images produce.

Images have different meanings. Some images are more expressively oriented than other images, which are more utilitarian. Compare, for instance, the kinds of images produced in the fine arts with the informational, schematic, and notational images found in scientific visualisations. The expressive meaning of an image is what opens its symbolic interpretation. An image then ranges from being more informational and utilitarian, to more symbolic and expressive.

The points that an image connects need not always be visible. Compare the hyper-visible images of pornography with the more subtle images of erotica. Pornogaphic images gives direct representation to sexualised content; erotica makes the absence of visibility salient.



Vision: How do we see images?


Vision occurs in optical perception, where the visible spectrum is read through the organs of sight. Perception is visual, optical and ocular. Nonetheless, to see is a cognitive act as much as it is a physiological one. Cognition and vision interact in a reciprocally informative way in the act of perception. As John Berger said, “every image embodies a way of seeing”, and every person sees an image in their own unique way.


Images pass quickly without us noticing the majority of them. Vision is composed by the visual system, that contains the visual field, the optical lens, and the retinal and brain processing. Think of one person observing an object and another observing it having taken hallucinogenic substances. Our perception partly builds reality, but this reality is not exhausted in our perceptions. Another example is colour blindness, which is the deficiency to see colour or differences in colour.


Example of an Ishihara colour test plate. The number "74" should be clearly visible to viewers with normal colour vision.


But seeing is not uniform, take for instance perception verbs like looking, paying attention, glimpsing, peeping, spying, noticing, observing, catching, comparing, picking out, detecting, having sight of, setting eyes on, surveilling, watching, keeping an eye on, shadowing, snooping, watching. All these forms of looking take different shapes when engaging with an image. Sometimes they result in reflexivity, attentiveness or awareness.

MIT neuroscientists find the brain can identify images seen for as little as 13 milliseconds, but they’re still uncertain of how long the images can be retained in the brain. In order to see properly there has to be time, slowing down the perception, re-imagining what the picture depicts, and allowing for contemplation. Furthermore understanding that the language of images does not equate to the language of text, and perhaps images can’t be fully read, but need also to be experienced.

Predictive coding models of cognition and perception argue that many of the images that we perceive are predictions: our brains make a best guess as to what the world contains, and then checks if the prediction is true. This means we often see what we expect to see instead of what’s actually there. This leads to phenomena like change blindness, where people do not notice even substantial changes to their sensory environment.


Seeing with the other senses


Though sight is often privileged as the dominant sense, there is good evidence that all the senses interact when representing the world. On the one hand, linguistic data shows that even abstract concepts are represented using human sensory and motor schemas that draw on all the senses. On the other, neuroscientific evidence shows that sensory impressions are cross-coded: the brain can predict tactile sensation from a visual input, and vice-versa. “Human brains capture and store physical sensations, and then replay them when prompted by viewing the corresponding visual image.”

Notably, blind people can repurpose their visual cortex for processing tactile data. This indicates that there may be processes common to all forms of perception.



Interactions between the visual and the tactile senses become more salient when the scale of viewing is increased. For example, Vermeer’s ‘Girl With a Pearl Earring’ has been represented as a 10-Billion pixel interactive image. Or Pluto’s true colors taken by NASA's New Horizons spacecraft. From a distance, seeing is optical, but the closer you get, the more tactile it gets. “Haptic visuality” refers to a visuality that has tactile effects, triggering further memories like smell, touch and taste. We touch what we see with our eyes.




In terms of sound, sight and hearing cooperate in spatial awareness. The brain’s (reasonable) assumption that sensory information will cohere can lead to counterintuitive outcomes. See, for instance, the McGurk effect where the incongruence of visual and auditory speech is not noticed and results in hearing something the voice is not saying. Another way of putting it is how music is represented visually.


The Magic Circle of Infinity from George Crumb's Makrokosmos


Kiki and Bouba
The cross-mapping of visual and auditory stimuli is grounded in very fundamental aspects of human perception. Take the Kiki-Bouba effect: across every culture in which it has been tested, individuals associate the nonsense word ‘Bouba’ with a round shape and the word ‘Kiki’ with a jagged shape. This indicates that we cannot easily consider images without bringing in other sensory modalities.

The visual and the visible: How does our social and cultural context shape our reception of images?



Vision refers to the act of seeing (which is perceptual), but what about what is seen? Vision and visuality are not the same, the first is a physical activity and the second is a socio-cultural construction.

Visuality and visibility are the complement to visual perception. These terms relate to our social and cultural relations with the visible world. Visuality has to do with lived experiences and social relations in the current technological ecosystem. Visuality therefore is the culturally and socially coded way in which we read the world. Looking and being looked at, knowing that the way we look (our gaze) is coded. Our gaze is modified by attentive desire or fear.

In terms of visibility, pictures are sometimes assigned ‘magical’ powers. Some cultures are constructed around them (idolatry) and some cultures want to destroy them (iconoclasm). Because images exist, it doesn’t mean we can always see or access them. Social media platforms control what is visible (nudity amongst other regulations). Images are constantly being censored, hidden, blurred. Part of what we see is mediated by the image’s visibility, what is represented and presented to us.

Images permeate and influence culture and are therefore means for support (or resistance) of power. Think, for instance, about the manipulation of pictures in false news, or the mainstream representation of women's bodies. Add to this the invisibility of some subjects and cultures, opposed to those who get the power of being represented. Take for instance the Netflix documentary Disclosure, which outlines topics of transgender misrepresentation and invisibility.

Despite the inequality of visibility, contemporary culture is hyper visual. We are immersed and surrounded by images. Our daily lives are flooded by brand, commercial and self-imagery. Popular culture and spectacle is based on images and the resignification of them. Modes of control are also image-based, take for instance biometrics and surveillance. The human body no longer needs to travel to places to depict events, we have images produced by drones, microscopes, telescopes, security cameras, deep fake, and even by machine vision. Generative Adversarial Networks (GANs) are algorithms that can produce photorealistic images of people who will never exist.

Style-GAN generated images of non-existent people


Adam Harvey and Jules LaPlace, Megapixels. MegaPixels is an art and research project that investigates the ethics, origins, and individual privacy implications of face recognition image datasets and their role in the expansion of biometric surveillance technologies.

Conclusion


In this text we talk about images as physical surfaces and as mental imaginations. But this shouldn’t stop us talking about them. It helps us to engage with vision as a physiological and cognitive process. This text also outlines the way in which the senses cooperate and we can touch with our eyes or hear what an image is saying (even if we’re not always accurate). This allows us to engage with a visual world that is often cross-coded with other sensory modalities.

But vision is culturally and socially coded, which relates to visuality and visibility. Visuality is the intention we put in our gaze. Visibility is the amount of participation and representation something gets in our experience of culture and politics.

Jean Luc Godard said that art deals not with the image of reality but with the reality of the image. This is why we should always ask ourselves: what is the story of the image?
Cactus Disease. Hand pain with areas of skin elevation. This case is fictitious and the described condition is not a real diagnosis. The images in this case have been digitally altered. The case was originally published as one of Radiopaedia.org's April Fools' cases.

Further Reading


Visuality, visibility and vision
Images and technology
Pictures
Thinking with images