A sign that spells
Machinic concepts and the racial politics of generative AI
DOI:
https://doi.org/10.33621/jdsr.v6i440462Keywords:
generative artificial intelligence, critical artificial intelligence studies, generative AIAbstract
In this paper, we examine how generative artificial intelligence produces a new politics of visual culture. We focus on DALL·E and related machine learning models as an emergent approach to image-making that operates through the cultural technique of semantic compression. Semantic compression, we argue, is an inhuman and invisual technique, yet it is still caught in a paradox that is ironically all too human: the consistent reproduction of whiteness as a latent feature of dominant visual culture. We use Open AI’s failed efforts to “debias” their system as a critical opening to interrogate how DALL·E dissolves and reconstitutes politically and economically salient human concepts like race. This example vividly illustrates the stakes of the current moment of transformation, when so-called foundation models reconfigure the boundaries of visual culture and when “doing” anti-racism means deploying quick technical fixes to mitigate personal discomfort, or more importantly, potential commercial loss. We conclude by arguing that it simply does not suffice anymore to point out a lack – of data, of representation, of subjectivity – in machine learning systems when these systems are designed and understood to be complete representations of reality. The current shift towards foundation models, then, at the very least presents an opportunity to reflect on what is next, even if it is just a “new and better” kind of complicity.
References
Ahmed, S. (2012) On being included: Racism and diversity in institutional life, Duke University Press.
Amoore, L. (2022) ‘Machine learning political orders’, Review of International Studies 49(1), pp. 20-36. https://doi.org/10.1017/S0260210522000031
Amoore, L. (2021) ‘The deep border’, Political Geography. https://doi.org/10.1016/j.polgeo.2021.102547
Azar, M., Cox, G. and Impett, L. (2021) ‘Introduction: Ways of machine seeing’, AI & Society 36, pp. 1093-1104. https://doi.org/10.1007/s00146-020-01124-6
Bajohr, H. (2023.) ‘Dumb meaning: Machine learning and artificial semantics’, IMAGE 37(1), pp. 58-70. https://doi.org/10.1453/1614-0885-1-2023-15452
Beyer, K., Goldstein, J., Ramakrishnan, R. and Shaft, U. (1999), ‘When is “nearest neighbor” meaningful?’, ICDT’99: 7th International Conference. Jerusalem, Israel, January 10–12, 1999, pp. 217-235.
Birhane, A., Prabhu, V.U. and Kahembwe, E. (2021), ‘Multimodal datasets: Misogyny, pornography, and malignant stereotypes’, arXiv preprint 2110.01963. https://arxiv.org/pdf/2110.01963.pdf
Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S et al. (2021) ’On the opportunities and risks of foundation models’, arXiv preprint 2108.07258. https://arxiv.org/pdf/2108.07258.pdf
Chiang, T. (2023) ‘ChatGPT is a blurry JPEG of the web’, The New Yorker, Feb. 9. https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web
Crawford, K., Paglen, T. (2019) ‘Excavating AI: The politics of training sets for machine learning’. https://excavating.ai
Daston, L. (2015) ‘Epistemic images’, in Payne, A. (ed.) Vision and its instruments: Art, science, and technology in early modern Europe, The Pennsylvania State University Press, pp. 13–35.
Denton, E., Hanna, A., Amironesei, R., Smart, A., & Nicole, H. (2021) ‘On the genealogy of machine learning datasets: A critical history of ImageNet’, Big Data & Society 8(2). https://doi.org/10.1177/20539517211035955
Fanon, F. (2008) Black skin, white masks. Grove Press.
Galison, P. (2002) ‘Images scatter into data, data gather into images’, in Latour, B., Weibel, P. (eds.) Iconoclash: Beyond the image Wars in science, religion, and art, MIT Press.
Gershgorn, D. (2017) ‘The data that transformed AI research – and possibly the world’, Quartz, July 26. https://qz.com/1034972/the-data-that-changed-the-direction-of-ai-research-and-possibly-the-world
Haraway, D. (2004) ‘A manifesto for cyborgs: Science, technology and socialist-feminism in the 1980s’, in The Haraway Reader, Routledge, pp. 7–47.
Haraway, D. (1997) Modest_Witness@Second_Millennium.FemaleMan_Meets_OncoMouse: Feminism and Technoscience, Routledge.
Harvey, A., LaPlace, J. (2021) Exposing.ai. https://exposing.ai/
Hayles, N.K. (1999) How we became posthuman: Virtual bodies in cybernetics, literature, and informatics, University of Chicago Press.
Hooker, S. (2021) ‘Moving beyond “algorithmic bias is a data problem”’, Patterns 2(4). https://doi.org/10.1016/j.patter.2021.100241
Impett, L., Offert, F. (2023) ‘There is a digital art history’ Visual Resources 38(2). https://doi.org/10.1080/01973762.2024.2362466
Birhane, A. and Prabhu, V.U. (2021) ‘Large image datasets: A pyrrhic win for computer vision?’, IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1536-1546.
Ross, J., Irani, L., Silberman, M.S., Zaldivar, A. and Tomlinson, B. (2010) ’Who are the crowdworkers? Shifting demographics in Mechanical Turk’, CHI'10 extended abstracts on human factors in computing systems, pp. 2863-2872.
Jackson, Z.I. (2020) Becoming human: Matter and meaning in an antiblack world. NYU Press.
Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012) ‘ImageNet classification with deep convolutional neural networks’, Advances in neural information processing systems 25. https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
Latour, B. (2004) ‘Why has critique run out of steam? From matters of fact to matters of concern’, Critical Inquiry 30(2), pp. 225-248.
Luccioni, A. S., Akiki, C., Mitchell, M., and Jernite, Y. (2023) ‘Stable bias: Analyzing societal representations in diffusion models’, arXiv preprint 2303.11408. https://arxiv.org/abs/2303.11408
Macho, T. (2003) ‚Zeit und Zahl. Kalender und Zeitrechnung als Kulturtechniken‘ in Krämer S., Bredekamp, H. (eds.), Bild – Schrift – Zahl, Wilhelm Fink, pp. 179-192. http://www.thomasmacho.de/index.php?id=zeit-und-zahl
MacKenzie, A., Munster, A. (2019) ‘Platform seeing: Image ensembles and their invisualities’, Theory, Culture & Society 36, pp. 3-22. https://doi.org/10.1177/0263276419847508
Malevé, N. (2021) ‘On the data set’s ruins’, AI & Society 36(4), pp.1117-1131. https://doi.org/10.1007/s00146-020-01093-w
Millière, R. (2022) ‘Adversarial attacks on image generation with made-up words’, arXiv preprint 2208.04135. https://arxiv.org/pdf/2208.04135.pdf
Moreton-Robinson, A. (2021) Talkin'up to the white woman: Indigenous women and feminism, University of Minnesota Press.
Mostaque, E. (2022) Stable Diffusion public release. https://stability.ai/blog/stable-diffusion-public-release
Offert, F. (2021) ‘Latent deep space: Generative adversarial networks (GANs) in the sciences’, Media+Environment 3(2). https://mediaenviron.org/article/29905-latent-deep-space-generative-adversarial-networks-gans-in-the-sciences
Offert, F., Bell, P. (2021) ‘Perceptual bias and technical metapictures. Critical machine vision as a humanities challenge’, AI & Society 36, pp. 1133–1144. https://doi.org/10.1007/s00146-020-01058-z
Offert, F. (2023) ‘On the concept of history (in foundation models)’, IMAGE 37(1), pp. 121-134. https://doi.org/10.1453/1614-0885-1-2023-15462
OpenAI (2022a) DALL·E 2 preview – Risks and limitations. https://github.com/openai/dalle-2-preview/blob/main/system-card.md
OpenAI (2022b) Reducing bias and improving safety in DALL·E 2. https://openai.com/blog/reducing-bias-and-improving-safety-in-dall-e-2/
OpenAI (2023) DALL·E 3 system card. https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf
Phan, T. and Wark, S. (2021) ‘What personalisation can do for you! Or: How to do racial discrimination without ‘race’’, Culture Machine 20, pp. 1-29.
Raley, R. and Rhee, J. (2023) ‘Critical AI: A field in formation’, American Literature 95(2).
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal et al. (2021) ‘Learning transferable visual models from natural language supervision’, International Conference on Machine Learning (ICML), 8748–8763. https://proceedings.mlr.press/v139/radford21a/radford21a.pdf
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M. (2022) ‘Hierarchical text-conditional image generation with CLIP latents’, arXiv preprint 2204.06125. https://arxiv.org/abs/2204.06125
Robertson, A. (2024) ‘Google apologizes for ‘missing the mark’ after Gemini generated racially diverse Nazis’, The Verge, Feb. 21. https://www.theverge.com/2024/2/21/24079371/google-ai-gemini-generative-inaccurate-historical
Salvaggio, E. (2023) ‘Shining a light on shadow prompting’, Tech Policy Press, Oct. 19. https://www.techpolicy.press/shining-a-light-on-shadow-prompting/
Scheuerman, M.K., Hanna, A. and Denton, E. (2021) ‘Do datasets have politics? Disciplinary values in computer vision dataset development’, Proceedings of the ACM on human-computer interaction 5(CSCW2), pp. 1-37. https://doi.org/10.1145/3476058
Shannon, C.E. (1949) ‘Communication in the presence of noise’, Proceedings of the IRE 37(1), pp.10-21.
Smits, T. and Wevers, M. (2022) ‘The agency of computer vision models as optical instruments’, Visual Communication 21(2), pp.329-349. https://doi.org/10.1177/1470357221992097
Turing, A.M. (1950) ‘Computing machinery and intelligence’, Mind 59(236).
Weatherby, L. and Justie, B. (2022) ‘Indexical AI’, Critical Inquiry 48(2), pp. 381-415.
Weheliye, A.G. (2014) Habeas viscus: Racializing assemblages, biopolitics, and black feminist theories of the human, Duke University Press.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Fabian Offert, Thao Phan
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
