My next exploration was to generate still images. The prompt was: "A self-portrait of a woman, fantasy style, with animals in the backgroun(d), by Hayao Miyazaki." The results stunned me. The women in these images look ghosty and uncanny. They are human-like, but something does not click. “Disco Diffusion is not good at generating human faces yet,” one engineer said. This opened the Pandora box.

Fig 2. Images generated by Disco Diffusion, July 2022.

I then asked the generator to create “A realistic photo of a young biracial woman portrait.” The outcome was shocking. We see distorted humans with physical features that struggle to construct united beings. But what caught my attention was the homogeneity between these visuals—black eyebrows, dark curly hairs, thick lips, and similar skin tones. One image symbolically but also literally, I would argue, reveals the logic behind generating “a young biracial woman,” where black and white arms were tangled together. (Fig 3.) It made me wonder: is AI profiling humans?

Fig 3. Images generated by Disco Diffusion, July 2022.

In August 2022, I generated a series of high-resolution human portraits. All prompts were written as the following format: “A realistic photo of a [race and/or ethnicity] [gender] portrait” and four images were generated for each ethnicity group. Each image took about an hour to complete, and I generated 36 images. (Fig. 4 & 5) Examining these images, my findings include but are not limited to:

​

1. All Asians look lighter skin Asians and have small eyes and epicanthic folds

2. All white people have blonde hair

3. Middle Eastern women wear hijabs

4. Most of the biracial people have dark features​​​​​​

Fig 5 Selected results with cropped physical features and their originals without colored background. August 2022.

Fig 4. Images generated by Disco Diffusion, August 2022.

There is a lack of inclusion and diversity in these images. AI generator never generates South Asians when representing Asians, asserts that white people only/always have blonde hair, asserts that middle eastern women always wear hijabs, and implies that mixed race people are black and white. This observation inspired me to create Pro5ling, a performance art embodying the affair of profiling. I carved out facial features from those generated images (Fig. 5) and turned them into graphic and text scores. The performer is instructed to “come up with one short musical/sonic segment for each graphic based on these prompts.” Prompts include “eyes refer to duration. Imagine they’re staring at you. How long/short is each stare?”, “hair refers to texture. How does each of them inspire you to come up with a musical or sonic texture?”, among others. (Fig. 6)​​

Fig 6. Score excerpts from Pro5ling, 2022.

The materials were provided to the performer in three stages. Prompts were offered during first stage, then performer works with full portraits where they utilize the prompts as foundation to assign sonic identity to each character. The rest of the materials are provided during live performance, in which the fact that the materials were AI generated was revealed. This work incorporates a custom smart mirror where the performer’s own reflection is layered on top of the artificial portraits. Through transparentizing the process of decision making, this work highlights the charged biases upon how we judge each other through facial features. (Media. 1)

Media 1. Premiere of Pro5ling, September 2022.

Yet when investigating the generated faces further, I noticed details that are even more problematic. Many of these images show extra or miss out physical features—such as eyes and noses. But in the image of a “black woman” and a “biracial man,” instead of human-like features, the visuals resemble primates like a chimpanzee. (Fig. 7) It reveals the old tropes of pseudo scientific tropes that are used to fuel racist propaganda that has existed for centuries to justify slave labour and perpetuate exploitation driven by capitalism.

​

​Black people being associated with primates is not news to Safiya Umoja Noble, the author of Algorithms of Oppression: How Search Engines Reinforce Racism. In 2015, Google’s algorithm “automatically tagged African Americans as ‘apes’ and ‘animals’” [4] through facial recognition software in its own photo application. After Obama’s presidential election in 2009, photoshopped images of monkeys’ faces placed on Michelle Obama’s body were circulating on the internet. [5] Years later in 2015, Google engine still offered autosuggestions associating Michelle Obama with apes. [6] The Council of Conservative Citizens (CCC), an organization of white supremacist activists, “run pictures comparing the late pop singer Michael Jackson to an ape and referred to black people as ‘a retrograde species of humanity.’” [7]

Fig 7. A “black woman” (left) and a “biracial man” (right) generated by Disco Diffusion, August 2022.

“While we often think of terms such as ‘big data’ and ‘algorithms’ as being benign, neutral, or objective, they are anything but,” claimed Noble. “The people who make these decisions hold all types of values, many of which openly promote racism, sexism, and false notions of meritocracy, which is well documented in studies of Silicon Valley and other tech corridors.” [8] She also pointed out how incidents like those are often framed as “glitches” and how they “do not suggest that the organizing logics of the web could be broken but, rather, that these are occasional one-off moments when something goes terribly wrong with near-perfect systems.” [9] Digital media platforms are still “resoundingly characterized as ‘neutral technologies’ in the public domain and often, unfortunately, in academia.” [10]

In Black Bodies, White Science: Louis Agassiz's Slave Daguerreotypes, author Brian Wallis combs through the visual history of constructing racial types. In 1850, biologist Louis Agassiz, an immigrant from Switzerland and former faculty at Harvard University along with English settler Robert W. Gibbes, took photos of a group of African American slaves through daguerreotypes, the first photographic process available to the public. The subjects were mostly naked, and those photographs captured various angles of their bodies. Wallis wrote: “The daguerreotypes, which were taken for Agassiz in Columbia, South Carolina, in 1850, had two purposes, one nominally scientific, the other frankly political. They were designed to analyze the physical differences between European whites and African blacks, but at the same time they were meant to prove the superiority of the white race. Agassiz hoped to use the photographs as evidence to prove his theory of ‘separate creation,’ the idea that the various races of mankind were in fact separate species.” [11]​ Through standardizing the procedure of classifying, categorizing, cataloguing, and archiving, the white gaze is neutralized by “unprejudiced” scientific studies [12] and by the establishment of typological photography [13] that justify white supremacist ideology and the state of slavery. There is constant dehumanization of living beings that are deemed as the Other—pretty much everything except the so-called white race. Whiteness becomes the ultimate reference point that is masked by being invisible. And just as whiteness is often framed as neutral, AI technology is too under the same framework.

​

Fast forward to a few centuries later, it is apparent that this legacy continues. Collecting data, categorizing types, and referencing archives are how AI functions. But how neutral and objective the development of this technology can be when its foundation is literally built and continues to be built upon discriminatory and colonial agendas? And how unbiased and just can AI technology be when the initial data was already racially coded?

​

Sociologist Ruha Benjamin, who coined the term “the New Jim Code,” [14] pointed out that the influence of discriminatory technology like AI does not only exist in the digital space, but it also operates in the analogue world. 87 percent of the names listed in California gang database are Blacks or Latinxs, but many of them were just babies under 1 year old. Having a racially coded name predicts the profiling and surveillance an individual must navigate in daily encounters from the state, who added those names to the database due to their preconceived radicalized association.[15] “Codes are both reflective and predictive…More than stereotypes, codes act as narratives, telling us what to expect…Codes, in short, operate within powerful systems of meaning that render some things visible, others invisible, and create a vast array of distortions and dangers.” [16] And how the New Jim Code manifests itself is that tech designers—most of whom are White or Asian male [17]—would “encode judgments into technical systems but claim that the racist results of their designs are entirely exterior to the encoding process.” [18]

​

Disco Diffusion’s dehumanizing images, due to its “glitches,” as some may say, exposed the AI bias. The process of generative decisions was made transparent, and users witness how an AI application calculates humanity through a racist lens. Some then argue that an effective generation depends on sufficient prompts, that users should learn how to communicate with AI based on its logic. When I asked why Disco Diffusion is not good at generating human faces, Google’s AI Overview provides advice to remedy the issue. One of them is using “Negative Prompt,” where users tell AI what to avoid, for example, “ugly faces.” (Fig. 8) AI companies and enthusiasts published prompt tutorials online, and vague terms like “ugly,” “bad,” “poor,” are common ones. Other words that are more specific but highly subjective include “unattractive,” “repulsive,” “error,” and “genetic variation.” [19-21]