Recently, Facebook’s friend-suggestion algorithm suggested that I friend my hairdresser. I couldn’t think of any digital trail that would link us; it seemed as though it might be the physical proximity of our phones for half an hour every month, but Facebook doesn’t use location information to suggest friends. Increasingly, I’ve had a sense of such algorithms’ uncanny omniscience, as if they were beings capable of magical clairvoyance rather than ordinary, if data-intensive, feats of computation. In the same instant as I am a body sitting staring at my phone screen, I realize I am also a set of entries in an enormous inscrutable database, being algorithmically worked over.
More and more of life may be seen through this lens: Where there is a congruity of events that in a different age may have been thought of as coincidence or the whims of a divinity, I now assume it must have been produced by a magical algorithm that knows everything about me and what I do. Algorithmically based media forms are emerging to represent these new realms of experience, illustrating space, subjectivity, affect, violence, and more.
To think in terms of “landscape” is to think of land as an entity shaped by human activity. Shifts in how we represent landscapes suggest changes in the horizons of possibility
Video games are perhaps the best example. Filmmaker Harun Farocki’s Parallel series of short films charts the evolution of video-game graphics over the past few decades, through machinima-style clips of sequences taken from games being played unconventionally, with a voice-over describing their contours and limits. The fourth work in the series, drawn mostly from Grand Theft Auto IV, focuses on in-game encounters with nonplayer characters. In one sequence, the player’s avatar points his gun at a shopkeeper, causing the shopkeeper to run out of the shop. “The saleslady has a short memory,” the laconic voice-over tells us, as if these movements are somehow more than just procedural. “As soon as she’s outside the door, she forgets that the hero pointed a gun at her. She returns to the store, is threatened, and flees the store again.” Farocki lets this loop go on for nearly a minute.
The repetition of the cycle brings out elements of her comportment, certain tics, like the way she runs against the wall for a few seconds before lurching into motion in the correct direction. Something like a personality emerges from these glitches in the attempted verisimilitude. We can’t help but see ourselves in the shopkeeper. “If she is threatened, she must leave the store,” the voice-over explains. “When she is outside, she must return to the store again. This tragic constellation reveals to the hero the limitations of human freedom of action.”
By explaining the movements in terms of her “memory” as well as the algorithm that orders her world, Farocki suggests the shopkeeper might not be so different from us humans, who are similarly governed, by both the facts of our bodies and the systems we are embedded in. The systems have become both more omnipresent and more opaque. They describe reality and produce it, altering the world and shifting the primary perspective from which we view it. As simulation and reality become increasingly indistinguishable, video games can hint at the novel configurations of subjectivity emerging from our embeddedness in algorithms.
Video games render reality fundamentally differently than other media. Video-game simulations don’t serve as merely mimetic representations of reality; they try to model underlying processes. As an example, I’m thinking of a game that I played for long nights in my teenage years: SimCity 4. The original SimCity debuted in 1989, and the premise has remained more or less the same since the start. Mayors (as players are referred to) build and manage a city as they please, laying out roads, zoning land, creating schools, hospitals, police stations, setting taxes, managing budgets, dropping in parks, stadiums, casinos, responding to emergencies, and so on, with no explicit goals or ways to win.
The city is seen in isometric view, from top-down, growing, shifting its contours, extending into new suburbs here or there. Certain districts fall into disrepair; others grow into centers of business; others might turn from industrial areas into residential enclaves. All of this “action” occurs in response to the mayor’s plans and decisions. I was so engrossed in the pleasures that the game offered that, until the age of 16, my experience of cities was probably as much virtual as physical — and thus also my understanding of landscape, of commuting, of the movement of people, of bodies in space.
Though it works in novel ways, SimCity also fits within a long genealogy of representations of landscape in Western cultural production. To think in terms of “landscape” is to think of land as an entity shaped by human activity. Shifts in how we represent landscapes suggest changes in the horizons of possibility for how people can live on land. Art historian W.J.T. Mitchell has written that early Renaissance landscape painting helped create a territorial relationship to the land that is intimately tied up with the history of European capitalism and imperialism. In the 20th century, film brought a different attitude; through affordable prints, previously inaccessible worlds adorned living room walls, creating a new, mediated form of personal relation to the world.
Walter Benjamin claimed in “The Work of Art in the Age of Mechanical Reproduction” that “a different nature opens itself up to the camera than opens to the naked eye.” We might ask of simulations like SimCity: What kind of nature does the video game open up? If the land was once seen as the locus of sublime experience or as a resource to be owned and plundered, in SimCity it becomes the site of a choreography of subterranean economic flows, to be understood in terms of dynamic algorithmic models. The view of the land and our possible experiences on it are presented not as irreducibly distinct singularities but as commensurate components within a universal pattern, reducible finally to data.
SimCity naturalizes modeling itself, which cannot be fully captured by the human brain, as a structuring principle of the world
Will Wright, the game’s creator, in a 2001 interview describes SimCity as a consisting of two levels of modeling. At one level, the game developers model the ways cities work, how they evolve and grow and decay, with bodies flowing through and sustaining them, how they are built on and shape land and air and water. At the second, players model the game developers’ model, engaging in a similar kind of conceptual work. “As a player, a lot of what you’re trying to do is reverse engineer the simulation,” he says.
Much critical writing on SimCity has focused on omissions in the game developers’ modeling, noting, for example, the complete absence of race, that most basic structuring principle of North American cities, or its simplistic taxation model. These points are important, but to limit analysis of the SimCity series to all the ways its models are inaccurate or oversimplified is anachronistic: It treats players as though they were simply readers or viewers passively receiving the designers’ intentions — looking at a painting or reading a book rather than playing a game. Even if SimCity’s particular model is superficially incorrect, it can show us how we build our own models of the models we engage with through play, and how these form feedback loops that reshape how we understand ourselves. Video games not only suggest how players would navigate in the real world the processes the games simulate — as when politicians have, on many occasions, publicly played SimCity to try to demonstrate their fitness for office. They also reveal how adeptly players can leap over into the computer and begin to see operationally, in flows of data.
As a condition of playing, SimCity trains its mayors to engage in the work of modeling. Each of a player’s actions is one of a series of algorithmically executed events. Building a new highway will have a series of consequences that are computed by the model: commute times might decrease, changing the attractiveness of a neighborhood, in turn affecting the demographics of the entire city. The economic flows that compose the model become visible through experiments with these events, depicted through an array of charts and graphs which show noise pollution by neighborhood, rush hour traffic timings, the amount of green space per square kilometer, and so on. Based on these data visualizations, the player — like their real-world analogue, the technocratic mayor — makes plans for the future, lays down a highway here, or rezones a tract of land there.
In a way, these visualizations are similar to the images in Farocki’s Eye/Machine, which are produced by bombs and drones and other high-tech automated weaponry for their operators. Artist Trevor Paglen, in an essay on e-flux, has described these as “operational images that have been configured by machines to be interpretable by humans.” Like SimCity’s graphs, they are made to translate the data constantly being worked upon by algorithms into a form comprehensible for human eyes. In this translation from the machine-processable to the human-viewable — a process Lev Manovich refers to as transcoding — some of the richness of the algorithm’s process is inevitably lost. An algorithm might make a million calculations in a second. The skill that the game cultivates in players is the ability, based on abstracted images, to approximate the outcomes of those calculations.
By involving players in this sort of experimentation, second-guessing algorithms, the game naturalizes modeling itself, which cannot be fully captured by the human brain, as a structuring principle of the world. In one of my favorite essays on SimCity, Ted Friedman suggests that “’losing yourself in a computer game’ means, in a sense, identifying with the simulation itself.” Above all, this identification, which is the sense that experienced players develop of the algorithm, is an immersion in a particular perspective: abstracted, data-driven, pattern-seeking, simultaneously aerial and subterranean. To use a phrase from Donna Haraway, this is a way of seeing “everything from nowhere,” a god trick, a false claim to objectivity.
At stake is not whether or not players think that this is how cities actually function — whether SimCity’s model “reads” as representationally accurate. The more pernicious issue is that players will come to think of this perspective as objective — if a particular model seems off, then it is only a matter of shifting certain parameters to make the model correct. In this view, the entire world works according to a hidden logic that can be captured by ever more precise algorithms. But the work of modeling will never be complete. What Kate Crawford writes about big data applies equally to algorithmic modeling: “If the big-data fundamentalists argue that more data is inherently better, closer to the truth, then there is no point in their theology at which enough is enough.”
Identification with a simulation is an immersion in a particular perspective: abstracted, data-driven, pattern-seeking; a way of seeing “everything from nowhere”
Most worryingly, models are not only describing a reality in SimCity; they are also projecting a reality in the world, and bringing it into being. In the world of the algorithmic god trick, everything about the city is expressible on the same universal field. Citizens are reduced to a set of numbers: income, level of education, health, for example. The world is another set of numbers: traffic (expressed as average commute time), pollution, amount of resources, for example. The city becomes no more than the complicated interaction between the two, a machine that takes as its inputs the citizen and the world and from them produces profit. What the model misses becomes a feature assuring this kind of economic efficiency.
The operative principle of SimCity — the algorithmic god-eye view — can be seen in action in the rapid changes taking place in actual cities. But rather than the suburban North American ur-city on which SimCity is based, it’s most stark in the instant cities of Asia, like the new so-called smart cities being instituted around India, or in Shenzhen, a metropolis with over 18 million people that in 1979 was a fishing village of 30,000.
The model of urban growth in SimCity is, on the surface, fairly simple. You lay out zoning for low-density housing, dirty industry, and small commercial businesses; you build electricity, water, and sewage infrastructure. As your city grows, you first expand it outward, building highways and transportation infrastructure to ensure the smooth flow of people and services, and schools, hospitals, and parks to make the city attractive to new migrants. People move in, built-up areas are rezoned for higher density constructions, a subway system is tunneled out. The implicit telos of the game, and of the world it describes, is the creation of a well-functioning economy, ideally accompanied by a downtown area with skyscrapers, high-density housing and commercial office spaces.
What’s striking is how closely Shenzhen’s growth has mirrored this process. In 1979, Deng Xiaoping, then the premier of the People’s Republic of China, picked the fishing village to be a Special Economic Zone. By 1986, a development master plan was put in place which concentrated urban growth in clusters along three highways. Ten years later, another development master plan aimed to curb low-density sprawl by restricting the amount of land which the city could spread into and investing in more highways and a massive subway project. The third development master plan, of 2005, designated a huge swathes low-density development as urban regeneration areas. This timeline would be familiar to any SimCity mayor. Rather than proving the accuracy of the SimCity model, the similarity shows how simulation and reality are co-produced, for the same processes and the modes of vision undergird both.
It is not surprising, then, that Shenzhen resembles most SimCities. When seen from the sky — the established perspective for urban planning — it is a group of commercial skyscrapers surrounded by an endless sea of high-rise apartment buildings, with green spaces and enormous superhighways interspersed between them. This design facilitates good connectivity, decent quality of life, and, in an area of extremely high populations, a relatively contained urban spread. It ticks off the indicators for what a city should provide its citizens while maximizing productivity.
Productivity is essential. The lived reality is often starkly different than what the indicators would suggest. The model fails to account for a lot: poor working conditions, brutal hours, high anxiety and stress levels, workers’ suicides. Yet the city is one of the central nodes in global capitalism, the site of enormous ports, huge shopping malls, and thousands of factories whose workers produce everything everything from knockoff designer handbags to the components in the electronic device on which you are reading this essay.
In most Renaissance landscapes, there is a human figure, or at least a sign of human habitation. Implicitly, the landscape unfolds in relation to this point. This is the tear in the canvas, so to speak, that gives the viewer a point of entry into the painting.
But where is the tear in SimCity 4? We watch sims walk down the street, into their cars, to their jobs, maybe stopping at a medium-density commercial building on their way home. There’s a shuttling of perspectives available, between the god-view of the mayor and the possibility of being a byte-sized citizen. The game allows us to rename the figures, to create families, to give them jobs, and imagine their lives as they advance in the career and through the city.
In the world of the algorithmic “god trick,” the city becomes a machine that takes as its inputs the citizen and the world and from them produces profit
The moments, then, when we see the extent to which these sims are mere emanations of data are jarring. If you watch the little characters carefully, their movements don’t make sense. They walk down to the end of a street and disappear. The game’s algorithm does not in fact model each sim; this would be too data-intensive. Instead it works through the data at the level of the entire city. When you build a new highway, each sim’s route to work is not recalculated; rather, the calculation is made at the level of neighborhoods, with clusters of sims. Then the tiny figures are extrapolated out of the data, which is why, when they come to the end of the street they so often disappear back into the data. At the instant before they disappear, we see them from two perspectives, just as Farocki’s video showed us the salesperson: as characters with inner lives and also as chunks of data being algorithmically processed.
In cities like Shenzhen, the residents are in effect also confronted with similar limitations. They bump up constantly against the limitations of a city and system designed at an inhuman scale. Their lives are treated as side-effects of the algorithm.
When SimCity 5 was released, in 2013, the game no longer had such jarring moments of double-identification; the developers were able to compute each and every sim. The sims no longer disappeared at the end of the street. But SimCity 5 required players to be online at all times so that Electronic Arts could collect data on their usage patterns, which would be used to render them into a tractable data set. What was inside the video game became reality, as it had in Shenzhen. For Electronic Arts, as for Facebook, Google and the Chinese state, we humans are accessories to larger systems structured for profit.
Last year I moved back from New York to New Delhi, where traffic rules are followed very loosely, and the cars weave in and out wildly between each other. I sometimes feel afraid. I don’t want to die in traffic. But on occasion, I have caught myself thinking about this less in terms of personal safety than economic productivity: Wouldn’t it just be more efficient if everyone drove in lanes? People would get to work faster, there would be fewer accidents, and goods would get to factories on time. Without intending to, I had fallen into exactly how I would have approached the city as a SimCity mayor. It is precisely this simultaneity of subjectivity — seeing myself both as a body with a life that can be extinguished at any moment and as a little generator of economic value moving through a vast system — that SimCity has made me aware of.
These two subjectivities will not feel incommensurable for long. The distance between them is collapsing. The moments of awareness that led me to write this essay are being smoothed over, glitches in the system are being troubleshooted. But their incongruity is important. Once I truly believe that efficiency is the compelling reason for people to follow traffic rules, I acquiesce to a world in which my own existence is a side effect of efficiency. The day we are no longer surprised when Facebook makes an uncannily accurate friend suggestion, we become that much more reconciled to seeing our social lives as data by-products.
The everyday consciousness of ourselves as simultaneously modeling and modeled might instead be a productive site from which to imagine alternative cities and political structures. Haraway argues for “a view from a body, always a complex, contradictory, structuring and structured body, versus the view from above, from nowhere, from simplicity.” This begins with an acknowledgement of our own complicities and opens out a possible world that is not reducible to data — where algorithmic models and the imperatives of profit are only a few among many ways of ordering space.
This essay is part of a collection on the theme of OBJECTIVITY. Also from this week, Linda Besner on false neutrality of tech and charities, and Anna Reser and Leila McNeill on the view from nowhere in approaches to climate change.