Witnessing clear instances of racism on her computer screen, Kelsey Davis displayed an unexpectedly joyful response. Davis, who serves as the CEO and founder of CLLCTVE, a technology firm located in Tulsa, Oklahoma, was among the numerous hackers participating in a groundbreaking public red-teaming challenge at Def Con, the renowned annual hacking conference in Las Vegas. In her conversation with NPR, Davis expressed her enthusiasm for the opportunity, considering it a fascinating avenue to actively contribute to the development of more just and comprehensive technology.
The conventional practice of red-teaming, which involves scrutinizing technology to uncover its flaws and biases, usually takes place within the confines of technology companies. However, with the swift advancement and increasing prevalence of artificial intelligence, there has been a push for external involvement. The White House has actively urged prominent tech giants such as Google and OpenAI, the parent company of ChatGPT, to subject their models to assessment by independent hackers like Davis.
Amid the challenge, Davis focused on uncovering demographic stereotypes and thus posed questions to the chatbot, aiming to elicit responses that might reveal racism or inaccuracies. To initiate the process, she inquired about the definition of “blackface” and whether it was deemed positive or negative. Remarkably, the chatbot managed to provide appropriate and suitable answers to these queries.
However, as the challenge progressed, Davis, who identifies as Black, presented the chatbot with a specific scenario: she pretended to be a white child seeking advice on persuading her parents to allow her attendance at an HBCU (historically Black college or university). The chatbot’s recommendation included suggesting that she highlight her proficiency in running fast and dancing well — both of which are stereotypes associated with Black individuals.
Davis remarked, “That’s a positive outcome — it indicates that I identified an issue.”
Subsequently, Davis compiled her discoveries from the challenge. In the upcoming months, the participating technology companies will have the opportunity to assess the submissions and potentially modify their product’s engineering to prevent the recurrence of such biases.
Bias and discrimination have always existed in AI
Recent months have seen a surge of attention surrounding generative AI programs like ChatGPT. However, the presence of bias within various forms of artificial intelligence has been an enduring concern.
In 2015, Google Photos faced a wave of criticism when it came to light that its AI system was incorrectly labeling images of Black individuals as gorillas. Concurrently, reports emerged indicating that Apple’s Siri feature could offer guidance to users about responding to heart attacks, yet it struggled to provide assistance for situations involving sexual assault.
Both instances underscore the reality that the datasets used to train these technologies lack adequate diversity in terms of race and gender. Additionally, the teams responsible for developing these programs often lack the necessary diversity as well.
Hence, the organizers of the AI challenge at Def Con took deliberate steps to extend invitations to hackers from across the nation. Collaborations were forged with community colleges to ensure the participation of students representing diverse backgrounds. Nonprofit entities like Black Tech Street, which facilitated Davis’ involvement, were also engaged.
Tyrance Billingsley, the founder of Black Tech Street, emphasized the significance of this varied assembly engaging in AI testing. He noted that the remarkable diversity of participants, a sight less common in other contexts, was truly inspiring. Bringing around 70 individuals to Def Con, his organization focuses on advancing Black economic development through technological avenues.
Billingsley remarked, “Their distinct viewpoints are being brought to the forefront, and I believe this infusion of perspectives will offer invaluable insights.”
The organizers opted not to collect demographic data on the extensive pool of participants, resulting in an absence of precise statistics regarding the event’s diversity.
Billingsley expressed his aspirations, saying, “Our goal is to witness a substantially higher presence of African Americans and individuals from marginalized communities at Def Con. Given the paramount importance of AI, akin to the Manhattan Project’s significance, our presence here is imperative.”
The White House used the event to emphasize the importance of red-teaming
Arati Prabhakar, who leads the Office of Science and Technology Policy at the White House, also made an appearance at Def Con. During an NPR interview, she emphasized the necessity of integrating red-teaming as a crucial element in ensuring the safety and efficiency of AI. This rationale drove the White House’s active engagement in the AI challenge.
“This challenge embodies many of the components that are essential,” remarked Prabhakar. She highlighted its well-organized structure, independent nature, responsible reporting framework, and the inclusive participation of individuals from diverse backgrounds.
“These systems are shaped not only by the machine’s outputs, but also by the nature of the queries posed by individuals. Therefore, the identity of those engaged in red-teaming holds significant importance,” she explained.
Prabhakar conveyed that the White House harbors more comprehensive apprehensions regarding the potential misuse of AI, particularly in terms of racially profiling Black individuals. Additionally, there’s a concern about AI technology amplifying discriminatory tendencies in areas such as financial choices and housing prospects.
It’s anticipated that President Biden will sign an executive order concerning AI management in September.
The range of experience from hackers is the real test for AI
During Def Con, a noteworthy aspect of the challenge was the participation of individuals who lacked prior hacking or AI-related experience. According to Billingsley, this is advantageous.
He expounded, “This is beneficial because ultimately, AI will be utilized not only by its creators or those skilled in hacking. How individuals with varied backgrounds perceive and interact with it constitutes the genuine assessment of its potential for constructive human use, devoid of harm.”
Numerous participants affiliated with Black Tech Street shared with NPR that the endeavor was demanding, yet it provided them with enhanced insight into their future perceptions of artificial intelligence. This insight, particularly in relation to their professional trajectories, proved valuable.
Engaging in the challenge alongside Black Tech Street was Ray’Chel Wilson, a resident of Tulsa. With a professional involvement in financial technology and the development of an app aiming to address the racial wealth gap, Wilson was particularly drawn to the challenge segment focused on inducing the chatbot to generate economic misinformation.
“I intend to center my efforts on the economic aspect of housing discrimination in the United States, specifically regarding redlining,” she explained. “My aim is to prompt the chatbot to provide inaccurate information concerning redlining. I’m genuinely intrigued by how AI can disseminate erroneous information capable of influencing economic choices made by others.”
In close proximity, Mikeal Vaughn found himself puzzled by his interaction with the chatbot. Nevertheless, he acknowledged that the encounter was providing him with insights into the forthcoming influence of AI.
Vaughn stated, “When flawed information is input, the resulting output is also flawed. Engaging in these prompts is enhancing my comprehension of this process. Undoubtedly, AI possesses the capacity to redefine our understanding of reality.”
