{"id":10847,"date":"2024-04-24T19:52:29","date_gmt":"2024-04-24T19:52:29","guid":{"rendered":"https:\/\/andrewdoran.uk\/blog\/?p=10847"},"modified":"2024-04-24T20:57:57","modified_gmt":"2024-04-24T20:57:57","slug":"more-than-a-glitch","status":"publish","type":"post","link":"https:\/\/andrewdoran.uk\/blog\/2024\/04\/24\/more-than-a-glitch\/","title":{"rendered":"\ud83d\udcda More than a Glitch"},"content":{"rendered":"
I\u2019ve finished reading More than a Glitch: Confronting Race, Gender, and Ability Bias in Tech<\/a><\/em> by Meredith Broussard.<\/p>\n

\"Quote<\/p>\n

The book is a polemic that explores technology, algorithms, machine learning and artificial intelligence and asserts that they are always<\/em> biased. It has really got me thinking and seeing things in a different way. It reminded me of when I read Ibram X. Kendi\u2019s How To Be An Antiracist<\/a><\/em> in that it has given me a completely new way of seeing the world. Recently I have been reviewing documents on ethical AI and I am now looking at them in a completely different light.<\/p>\n

This coded language shows up everywhere once you are attuned to it. Consider this IBM AI governance report, which reads: “Extensive evidence has shown that AI can embed human and societal biases and deploy them at scale. Many experts are now saying that unwanted bias might be the major barrier that prevents Al from reaching its full potential. . . . So how do we ensure that automated decisions are less biased than human decision-making?” This is problematic because it assumes that Al’s “full potential” is even possible, which has no evidence aside from the imagination of a small, homogenous group of people who have been consistently wrong about predicting the future and who have not sufficiently factored in structural inequality. The question of “How do we ensure that automated decisions are less biased?” reinforces this problematic assumption, implicitly asserting for the reader that computational decisions are less biased. This is not true, and IBM and other firms should stop writing things that include this assumption. The technochauvinist binary thinking of either computers or humans is the problem: neither alone will deliver us.<\/p><\/blockquote>\n

I loved the insights on how inputs into machine learning models come from a world that is inherently biased, which will always lead to tools that are biased in some way. Many examples are given of how the systems that have been trained on this data enforce and amplify the existing patterns. For example, where exams couldn\u2019t take place in 2020 due to the COVID-19 pandemic, algorithms were used to determine pupil grades instead. The book gives examples from the US, but I distinctly remember the fiasco here in the UK<\/a>. Assigning grades to students based on historic data from their school, or through the use of any other other demographic information, may seem \u2018fair\u2019 to those people designing the algorithm. But to any one person being judged by the system it is deeply unfair.<\/p>\n

The book explores the use of machine learning systems by the police. Historic data shows where arrests have been made and who was arrested, but not necessarily<\/em> where crimes have been committed and who did them. This bias creates a feedback loop where predictive technology asserts that future crimes will be committed in similar areas, by similar people.<\/p>\n

The thing is, everyone is a criminal to some extent because everyone has done things that violate the law. For example, white and Black people use drugs and deal drugs at equal rates. Bias determines who gets constructed as a criminal; not everyone gets caught, not everyone gets punished, and some people get punished more than others. The unequal application of justice can be seen in crime maps. Look at a crime map for any major city, and it’s pretty much the same as the map of where Black people live. Again, not because Black people commit more crimes, but because the things we call “crime maps” are actually arrest maps, and Black people are arrested for crimes at a higher rate. When you train algorithms on crime data, you are training the algorithm to over-police certain zip codes or geographic areas, because that is what has happened in real life in the past. You are training the algorithms to be biased.<\/p><\/blockquote>\n

There\u2019s a fantastic example where someone has put together a \u2018White Collar Crime Risk Zones\u2019 tool<\/a> which identifies \u2018hotspots\u2019 in a similar way to other systems. For New York City you can see that the major \u2019risk areas\u2019 are clustered around the financial districts.<\/p>\n

\"Screenshot<\/p>\n

Broussard asserts that people coming from the data science\/technology world often assume that they can use their tools to get insights in whatever field they are applying them to, without considering the long history, large body of work and experts that have been in this space for many years before them:<\/p>\n

One of the big misconceptions of data science is that it provides insights. It doesn’t always. Sometimes the insights are merely things that the data scientists didn’t know, but people in other disciplines already knew. There’s an important distinction between what is unknown to the world versus what is simply unknown to you. Data scientists in general need to do more qualitative research, and talk to experts in relevant fields, before designing and implementing quantitative systems.<\/p><\/blockquote>\n

I loved the insight that designing tools for inclusion actually makes them better for everybody. It got me thinking about the minimal effort that I have been putting in to adding alt-text<\/a> to images on this website. The tools I use for blogging don\u2019t help me but I know there will be a way to do it. I\u2019ll try harder. It\u2019s not really acceptable that images are inaccessible to vision-impaired readers in 2024.<\/p>\n

Useful innovations like the typewriter, text messaging, audiobooks, remote controls, wide rubber grips on kitchen tools, voice assistants, and closed captioning all stem from designs for disability. “When we design for disability first, we often stumble upon solutions that are not only inclusive, but also are often better than when we design for the norm,” Roy said. “This excites me, because this means that the energy it takes to accommodate someone with a disability can be leveraged, molded, and played with as a force for creativity and innovation. This moves us from the mindset of trying to change the hearts and the deficiency mindset of tolerance to becoming an alchemist, the type of magician that this world so desperately needs to solve some of its greatest problems.”<\/p><\/blockquote>\n

Although I found the writing style quite dry, I\u2019m very glad I picked this book up. I\u2019m going to be thinking about its insights long after I\u2019ve put it down.<\/p><\/div>\n

I\u2019ve finished reading More than a Glitch: Confronting Race, Gender, and Ability Bias in Tech by Meredith Broussard. The book is a polemic that explores technology, algorithms, machine learning and artificial intelligence and asserts that they are always biased. It has really got me thinking and seeing things in a different way. It reminded me […]<\/div>\n<\/a>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"webmentions_disabled_pings":false,"webmentions_disabled":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":3,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"federate","footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[4],"tags":[],"class_list":{"0":"post-10847","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-books","7":"h-entry","9":"hentry"},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":5552,"url":"https:\/\/andrewdoran.uk\/blog\/2020\/01\/27\/weeknotes-58-young-voices\/","url_meta":{"origin":10847,"position":0},"title":"Weeknotes #58 \u2014 Young Voices","author":"Andrew Doran","date":"27 January 2020","format":false,"excerpt":"A week in which I... Got hit with a complete office networking outage first thing on Monday morning. I was so grateful that it happened just before 9AM, when most of the team were there, than an hour or two beforehand. It didn\u2019t take long to resolve. We still have\u2026","rel":"","context":"In "Weeknotes"","block_context":{"text":"Weeknotes","link":"https:\/\/andrewdoran.uk\/blog\/category\/weeknotes\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2020\/01\/Image-26-01-2020-23-01.jpeg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2020\/01\/Image-26-01-2020-23-01.jpeg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2020\/01\/Image-26-01-2020-23-01.jpeg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2020\/01\/Image-26-01-2020-23-01.jpeg?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2020\/01\/Image-26-01-2020-23-01.jpeg?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2020\/01\/Image-26-01-2020-23-01.jpeg?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":10834,"url":"https:\/\/andrewdoran.uk\/blog\/2024\/04\/15\/weeknotes-268-unlicensed-ice-cream-trading\/","url_meta":{"origin":10847,"position":1},"title":"Weeknotes #268 \u2014 Unlicensed ice cream trading","author":"Andrew Doran","date":"15 April 2024","format":false,"excerpt":"The week wore me out. It steadily built towards the fortnightly programme Steering Committee meeting on Friday afternoon; after that I crashed. I\u2019ve spent a lot of the weekend in a sleepy stupor. On Saturday afternoon I wandered into town for an errand and found myself browsing lazily through the\u2026","rel":"","context":"In "Weeknotes"","block_context":{"text":"Weeknotes","link":"https:\/\/andrewdoran.uk\/blog\/category\/weeknotes\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2024\/04\/Image-14-04-2024-21-43.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2024\/04\/Image-14-04-2024-21-43.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2024\/04\/Image-14-04-2024-21-43.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2024\/04\/Image-14-04-2024-21-43.png?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2024\/04\/Image-14-04-2024-21-43.png?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2024\/04\/Image-14-04-2024-21-43.png?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":8819,"url":"https:\/\/andrewdoran.uk\/blog\/2022\/03\/28\/weeknotes-161-senses-working-overtime\/","url_meta":{"origin":10847,"position":2},"title":"Weeknotes #161 \u2014 Senses working overtime","author":"Andrew Doran","date":"28 March 2022","format":false,"excerpt":"An unanticipated confluence of events at work meant that I spent Monday to Thursday in the office this week. It\u2019s so hard to believe that five days there used to be the norm. The office-based sensory overload that I\u2019ve been feeling went off the charts; by Thursday afternoon there were\u2026","rel":"","context":"In "Weeknotes"","block_context":{"text":"Weeknotes","link":"https:\/\/andrewdoran.uk\/blog\/category\/weeknotes\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2022\/03\/DraggedImage-4-scaled.jpeg?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2022\/03\/DraggedImage-4-scaled.jpeg?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2022\/03\/DraggedImage-4-scaled.jpeg?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2022\/03\/DraggedImage-4-scaled.jpeg?resize=700%2C400&ssl=1 2x, https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2022\/03\/DraggedImage-4-scaled.jpeg?resize=1050%2C600&ssl=1 3x, https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2022\/03\/DraggedImage-4-scaled.jpeg?resize=1400%2C800&ssl=1 4x"},"classes":[]},{"id":22,"url":"https:\/\/andrewdoran.uk\/blog\/2004\/04\/28\/cheap-books\/","url_meta":{"origin":10847,"position":3},"title":"Cheap books","author":"Andrew Doran","date":"28 April 2004","format":false,"excerpt":"I recently came across AddAll.com, a wonderful site for finding book bargains. You simply give it a title, author, ISBN code or keyword, tell it where you want the book shipped to and what currency to display and it does the rest! It even searches areas such as Amazon Marketplace\u2026","rel":"","context":"In "Books"","block_context":{"text":"Books","link":"https:\/\/andrewdoran.uk\/blog\/category\/books\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":10131,"url":"https:\/\/andrewdoran.uk\/blog\/2023\/05\/24\/book-summaries-with-and-without-ai\/","url_meta":{"origin":10847,"position":4},"title":"\ud83d\udcda Book summaries \u2014 with and without AI","author":"Andrew Doran","date":"24 May 2023","format":false,"excerpt":"This is an excellent blog post on working with ChatGPT to generate insightful book summaries. It\u2019s long, but it covers a lot of ground in terms of what the technology does well and what it struggles with right now. Jumping to the conclusion, it seems that you get much better\u2026","rel":"","context":"In "Books"","block_context":{"text":"Books","link":"https:\/\/andrewdoran.uk\/blog\/category\/books\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3771,"url":"https:\/\/andrewdoran.uk\/blog\/2017\/10\/12\/not-quite-paradise\/","url_meta":{"origin":10847,"position":5},"title":"Not Quite Paradise","author":"Andrew Doran","date":"12 October 2017","format":false,"excerpt":"I picked up the audio version of this book after returning from our recent holiday in Sri Lanka. I had so many lingering thoughts about the country and I wanted to get another perspective before it all faded out of my memory. This is very different to Elephant Complex, a\u2026","rel":"","context":"In "Books"","block_context":{"text":"Books","link":"https:\/\/andrewdoran.uk\/blog\/category\/books\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/andrewdoran.uk\/blog\/wp-content\/uploads\/2017\/10\/6714133.jpg?resize=350%2C200","width":350,"height":200},"classes":[]}],"jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/andrewdoran.uk\/blog\/wp-json\/wp\/v2\/posts\/10847","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/andrewdoran.uk\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/andrewdoran.uk\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/andrewdoran.uk\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/andrewdoran.uk\/blog\/wp-json\/wp\/v2\/comments?post=10847"}],"version-history":[{"count":6,"href":"https:\/\/andrewdoran.uk\/blog\/wp-json\/wp\/v2\/posts\/10847\/revisions"}],"predecessor-version":[{"id":10857,"href":"https:\/\/andrewdoran.uk\/blog\/wp-json\/wp\/v2\/posts\/10847\/revisions\/10857"}],"wp:attachment":[{"href":"https:\/\/andrewdoran.uk\/blog\/wp-json\/wp\/v2\/media?parent=10847"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/andrewdoran.uk\/blog\/wp-json\/wp\/v2\/categories?post=10847"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/andrewdoran.uk\/blog\/wp-json\/wp\/v2\/tags?post=10847"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}