Month: October 2022

Social semiotics: a theory of gaze

What was most surprising about a recent ‘interdisciplinary’ LSE workshop on Attention, organised by the Anthropology department, was the absence of sociology as well as media and communications researchers; most participants were either anthropologists or psychologists.

You would expect that media and communications which is primarily concerned with media industry, media text, and media audience would be interested in the notion of attention as a prerequisite to any act of mediation or communication. Also, if sociology is ultimately about meaningful social action, attention must be seen as the first phase of any social action.

The truth is that there is no social account of attention developer by social theorists; an account which can explain the initial orientation which makes any social action possible. Can we speak, listen, read, write, love or hate without an orientation toward another person or institution? Is it possible to imagine a social act without an orientation toward a social actor?

This initial orientation, also known as attention, is the foundation for any relation between social objects, i.e. actors, institutions, or artefacts, which we call social action.

Before any social action, there is a relation between an attender and an attendee which we call attention. This foundational notion to the social world can be called social semiotics.

Gaze capital, gaze relations

Here I would like to propose a different concept instead of attention to extend this idea of social semiotics. A concept which will be particularly useful within media and communications but can also be applied to sociology.

Gaze is a an inherently relational concept referring to the link between a gazer and the gazed. What gives gaze an advantage over attention is its critical nature. Gaze is already theorized as a theory of power relations by various scholars such as Lacan, Foucault, and Mulvey. While they mostly see gaze as negative, a disempowering or rarefying relationship, they open up the possibility of a positive kind of gaze which empowers and enables the gazed.

It is the power dimension of the notion of gaze which makes it a perfect tool to theorise media and communications. All communication is gaze, I claim. Gaze in this sense is not limited to what eyes do when they look at something or someone; it is a a focused attention embedded in a power relation. Thus, it includes any act of focused listening, smelling, touching, etc. Ironically, humans and other animals with vision, usually accompany gaze when using other senses. When we hear someone calling our name, we immediately and unintentionally look at the direction of the sound. That’s also true when something touches us or when we smell something; we always orient our vision toward our perceived source of that sense. Even blind people always orient themselves toward the source of sounds, as if listening is impossible without gazing.

If the media text is built upon relations of meaning between symbols, reception of that media text happens in a gaze relation. When you read or watch or listen to a media text, you establish a gaze and at the same time a power relation with that text.

Here I need to introduce the idea of gaze capital to be able to explain negative or positive gaze.

Gaze capital is the sum of all negative or positive gazes a person or even an object have accumulated. For instance, celebrities are those who have received millions of gazes from other people with little gaze capital. That’s the very essence of their power. By the same logic, a popular website is a website which has received a lot of gazes from other websites through hyperlinks. So is a viral social media post. Viral means a media text which is continuously gaining gaze capital.

Negative and positive gaze

Now the tricky question is what differentiates negative from positive gaze. When does a gaze disempower the gazed and when does it empower it?

I propose that a negative gaze happens when the gazer has immensely more gaze capital than the gazed. Positive gaze then will be when the gazer has vastly less gaze capital than the gazed. In other words, it is the inequality of gaze capital which determines the nature of the power relation underlying the very gaze relation.

With this definition, when an ordinary man in the streets gazes at a female celebrity it will not objectify or disempower the celebrity. However, when an ordinary man gazes at another ordinary woman with roughly the same gaze capital in the metro, it will make the woman uncomfortable and objectified and powerless.

On screen, a female actress will not be disempowered by millions of film viewers whose individual gaze capital is massively less than the collective gaze that the actress on the screen is accumulating. But she will feel disempowered if a more famous male director gazes at her on the set. the same disempowerment will not necessarily be the case when the cameraman gazes via the camera lens at her

This may explain why major celebrities do not bother much if their sex tapes or private photos are leaked as long as they are not gazed at by more powerful gazers such as the law enforcement institutions.

A complex example of negative vs. positive gaze happens online. The difference with the offline world or non-interactive media forms such as television is online gazes are multifaceted. For instance, when someone leaves a comment or clicks on the Like button, they leave a trace of their gaze which can itself be gazed at by others.

The gaze theory can still explain this: Hurtful comments that ordinary people (i.e. people with low gaze capital) under other ordinary people are often painful to the original author, especially when they are algorithmically given more visibility, which means more ordinary people gaze at them.

That is also the case when famous online figures, with major gaze capital, face thousands of hurtful comments under a post. They often do not care and sometimes they somewhat enjoy the attention, despite its malign nature. But when one of these comments gains more gaze than others through the platform’s algorithmic intervention, the authors may feel threatened so they either erase or hide or reply to that comment.

Quoting other tweets creates similar but more complex dynamics. With this theory of gaze capital and gaze relations, quoting is a two-fold gaze: first is the gaze of the quoter at the quoted tweet, second is the author’s invitation to her followers to gaze at that tweet by another author. Each of these sorts of gazes generate a different power relation whose mutual impact ultimately determines the final power relation.

Gaze, data and platforms

Having introduced concepts such as gaze relation and gaze capital, I can now define mediation as gaze relation and the media as systems where these gaze relations are negotiated and ultimately regulated. A newspaper is where gaze relations between the producers, audiences, and texts are negotiated, configured, and regulated.

If conventional media are systems (or figurations) of gaze regulation, the digital platforms add data relations to this figuration. The data relations which platforms systematically establish between users, producers, advertisers, etc. through surveillance and classifications will be used to regulate the gaze relations between them. Thus, platforms can be seen as where data and gaze relations are regulated through complex automated technologies.

Conclusion: a new model of communication?

Communication, and thus media, is a social practice which has so far been mostly theorised around psychological accounts of signs and thereby semiotics. The transmission model, including its more advanced versions such encoding/decoding, still views mediation as a semiotics system between signs and interpreters of signs. The gaze theory, proposed in this article, generates a social account of communication which happens between human subjects; thus I call it a social semiotics. Concepts such as gaze relations and gaze capital can enhance our understanding of media and explain the complexities that are emerging in the platform society.

— —

*Hossein Derakhshan is a PhD researcher at LSE.

Mass personalization of truth

It was five months into my eight-month solitary confinement when I heard it. It was a week before the Persian new year, Nowruz, and the guards had just put me in a new cell at the other end of the Iranian Revolutionary Guard Corps facility in Evin prison in Tehran. It was much larger than my old one, perhaps three by three square metres, which meant I could walk in a figure of eight between the corners. It was less bright though, given that it was close to the main entrance with its tall plane and mulberry trees. There were four horizontal metal bars welded to each tall window, which were angled slightly downwards, so you could see nothing but the sky and those beautiful trees.

Despite my numerous appeals, I was only allowed a single book in my cell: the Quran. By then, I had read it cover to cover, despite its tweet-style format, a few times. My only amusement, the interrogations, had long been concluded and I had only my own thoughts to entertain me. I constantly walked and spoke to myself, while looking up at the slices of sky and trees through the windows or at the half-marble-covered walls, which I discovered were filled with amazing patterns.

Sometimes there were sounds, too, especially when the heater was off. I could hear the guards speaking to other inmates or to each other. A few magical moments also occurred when the guards watched TV in their room on this side of our “ward”. They were strictly advised not to let me know anything about the news, but sometimes they watched other things and I heard bits of music from commercials or other shows. If you knew how closed the Iranian state media were to Western music, you might be less surprised about how a few seconds of Yann Tiersen’s tacky “Comptine d’un autre été, l’après-midi” could make me cry with joy. This was how isolated I was, physically and emotionally.

One afternoon, though, I heard something even more magical. Four young inmates were in a cell two down from me. (You could tell by the number of inmates by how many slippers were left outside each cell.) Through the ventilation shafts that connected the cells, I heard a newspaper rustling, a most amazing sound that truly melted my heart. The guards and interrogators had always said no one was given books or newspapers in our ward. I had believed them, because I had had no sight (nor heard any sound) of them.

Of all injustices of a high-security prison ward, from the blindfolded walking breaks in the yard to the awful grey polyester uniform and the cheap blue nylon underwear, this one felt the harshest. We were all equally subject to those, but as a journalist, not having a permit to read newspapers added another layer, which was the most painful. A decade later, amid the global debate on data, algorithms, and the new world they’re making, the term “filter bubble” keeps reminding me of those memories.

The hypothesis that people are totally confined in infor- mation bubbles has been discredited by researchers in the past couple of years. Evidence shows that people’s beliefs have little to do with their level of exposure to difference or dissent. Quite the opposite: people not only expose themselves to a range of different ideas and messages, perhaps out of curiosity, but are also much more open to some of them than we assume. However, these kinds of concerns inspire some profound questions.

What if there were no ventilation shafts? What if the ward were so vast that we never felt the presence of others? What if they could make us deaf as they made us blind? What if they could enclose our senses as they did our bodies? A wider question emerges: what is the condition of possibility of justice?

A market of one used to be the dream of marketers around the world. Digital platforms like Facebook and TikTok made it come true through what is now known as mass personalisation: the automated, continuous process of hyper-fragmenting consumers and predicting their needs or desires based on massive data surveil- lance and complex technologies of classification.

Businesses report a significant increase in sales when they use personalised marketing technologies, and political campaigners seem happy to spend money on targeted advertising. Nearly 60% of Amazon’s sale conversions come from personalised recommendations. Data shows personalisation drives 5 to 15% increase in revenue and raises 10 to 30% marketing-spend efficiency. Mass personalised delivery of goods, services, and messages has now become a ubiquitous reality. From Facebook, Instagram, and Twitter feeds and their embedded adverts to Amazon and Netflix recommendations and Spotify’s Weekly Discover playlist, sophisticated personalisation algorithms are at play to make them not only relevant to our daily lives, but also highly addictive. Using statistics and probability, they quickly learn what kind of things we may need or desire and nudge us towards them accordingly.

Many politicians and policymakers around the world have now fallen for a more radical idea: a society of one.

This requires a much deeper kind of mass personal- isation, something beyond personalised messages, goods or services. A society of one means the mass personalisation of truth.

Truth in this sense is different from reality. Unlike reality, truth is not just cognitive and private, but also sensory and material, as well as public or shared, and thereby social. Our realities deal with what we eat or read or watch at present, but our truths deal with our “gut feelings” about how things are, have been and will be. If reality is about cognitive short-term experiences, truth is about long-term affective meanings.

Mass personalisation of truth is where both our bodies and minds are affected by automated technologies of prediction and fragmentation. It is not just about listening to your weekly Spotify-curated playlist, but about listening to it through headphones and earbuds that in effect privatise your sensory and bodily experience, even in public spaces such as public transport. It’s not only about where Google Maps suggests we get a coffee, but also the route we should take to get there; it’s not only about showing you anti-smoking or pregnancy- related ads on Facebook, it’s about automated decisions as to whether you are qualified to receive a loan or to raise your private health-insurance premium.

The implications of the mass personalisation of truth are immense. It affects notions of trust, justice, and autonomy. When we live by personalised truths, our shared confidence in social institutions such as science, education, or law erodes. Trust is inherently social – who wants to fly with an unknown airline on an empty airplane? When there is no public space for shared truths to emerge, how do we even know whether we are being treated fairly by police, courts or our employers? The very notion of discrimination presupposes a prior knowledge of the situations of others. There will revolts in many companies if everyone’s salaries becomes known to others.

Moreover, when social systems can ever more accurately anticipate our life expectancy, health costs, education level or economic productivity, why would states or corporations abide by any universal allocation of resources, equal rights or ethics of care? Even some tax payers around the world may oppose policies which invest equally in people if they know their money will most likely be wasted.

When most of our future actions will be known to holders of big data, with a low margin of error, how autonomous would we really be in our decisions as agents of democratic systems? How can any notion of democracy be imagined without autonomous citizens? Perhaps the craziest of all is how the idea of politics could be hollowed out. If mass personalisation of truth leaves little or no space for shared experiences, what would stop politicians to champion opposite things for different groups of electorates? A politician can campaign on a racist platform for racist voters and run as an anti-racist for another group at the same time. If you find the idea absurd, ask a loved one about what kind of adverts they see on their social-media feeds.

A society of one may, in 2021, sound like an impossible dream (or nightmare, depending on who you are) – but so was the market of one before the emergence of giant digital platforms. The real threat of mass personalisation is not to our minds, but to our embodied truths. ◉

Published in Tank Magazine, Issue 88.