“Berkeley Law Voices Carry,” hosted by Gwyneth Shaw, is a podcast about how the school’s faculty, students, and staff are making an impact — in California, across the country, and around the world — through pathbreaking scholarship, hands-on legal training, and advocacy.
Listen to this conversation on the “Voices Carry” podcast.
This episode features the authors of a recent report from Berkeley’s Human Rights Center, “An International Analysis of the Human Rights Impacts of Large Language Models.” Human Rights Center Executive Director Betsy Popken wrote the report with two former technology researchers for the center: Vyoma Raman, who’s now pursuing a Ph.D. in information science at Cornell, and Camille Chabot, who just started her J.D. at Harvard Law School.
The report focuses on how the use of large language models (LLMs) — the foundation of generative AI — are affecting the fields of law, journalism, and education. Over 15 months, the team interviewed 56 people from 24 countries, including at least one expert and one practitioner in each sector on each inhabited continent.
The results are a window into what the benefits, and drawbacks, are for this technology in these sectors. And the authors conclude with a robust set of recommendations aimed at identifying and remediating potential harms in these important fields.
What follows is an edited version of the conversation. Listen to the full episode below and visit the “Voices Carry” archive for all episodes.
Gwyneth Shaw: How did the idea for this study come up?
Betsy Popken: For me, the idea really came up when ChatGPT was launched to the public at the end of 2022. I began thinking not just what the advantages were, but also what the potential risks might be. And I would say at that time, so much of it was focused on the opportunities versus the harms that could come as a result.
So I began thinking, what could we do as the Human Rights Center to really make an impact in helping people understand the risks and providing recommendations for stakeholders to mitigate those risks and maximize the opportunities? So I began thinking about that at the end of 2022 and into 2023. I then was connected with Vyoma and Camille because they were former Investigations Lab students of the Human Rights Center. We sat together and we decided how we would progress, looking at both the risks and then really wanting to incorporate Vyoma’s expertise as a computer scientist as well.
My background in this came from when I worked at a law firm, Orrick, Herrington & Sutcliffe. I co-founded and led the business and human rights practice at the firm, and many of our clients there were technology company clients.
And together we worked with those clients to understand the risks of their technology products and services and develop plans, both policies and procedures to help mitigate those risks. So that was the background through which I came to this project and working on this.
Camille Chabot: I had been thinking about the impact of new technologies on human rights for a few years working at the Human Rights Center, both in terms of investigations but also in terms of new harm, such as the use of drones, for example.
And so when ChatGPT came out, I remember thinking to myself that this new technology had both tremendous impact, but also a lot of risks associated with its use, especially in professional fields. And I was very attracted to the idea of conducting this kind of research in an interdisciplinary setting, which I had already experienced at the Human Rights Center, and the opportunity to collaborate with both Betsy, who had a lot of experience at a private firm, and with Vyoma, a computer scientist, and thinking through these questions alongside lawyers, experts, and professionals was really attractive to me.
Vyoma Raman: I was likewise very drawn to the opportunity to connect with Betsy and Camille about this topic because coming from more of the computer science side, one of the gaps that I felt that I had really seen is an understanding of how these tools are creating real grounded impact in different communities as opposed to simply being something that one puts out into the world and sees what happens next. And I think this is something that I find to be maybe a recurring problem or question in computer science spaces — how do we pick the problems we choose to solve? Whose problems matter? And why are we choosing to formulate problems in one way or another?
When you’re building an AI system, you are, especially with these modern forms of AI, optimizing it on a particular objective or goal based on a whole lot of data that is supposed to represent the world in some way. But there are challenges with both which data you use and who is represented and who is not represented, and also how exactly you formulate the objective and the goal.
This has been studied quite a lot in the algorithmic fairness literature at the level of the technical systems. I saw this project as an opportunity to go beyond that, to connect to the downstream human rights impact in a way that human-computer interaction scholars end up doing. I think working with Betsy and Camille was especially valuable because of their expertise in law and international relations that helped to ground exactly what the problems could be and design our methodology around excavating those because I think the design of the methodology can also drastically influence what you end up finding in these situations.
Gwyneth Shaw: Why was it important to study and analyze what’s happening given the perspectives that all of you were coming from?
Betsy Popken: For me, it was that so little attention was being paid to the risks. I think early on we saw analyses happening in the medical field because that was one where people could actually see the tangible risks of something going wrong, but we didn’t see it in any other professional field. I have a background in law and I’m an educator, and we also have a number of journalists on our team at the Human Rights Center, so those were three fields where we felt like we could really begin to understand what those risks might be, and we’d have kind of connections to people all around the world.
We were also interested as a team in understanding the different risks based upon how people use it in different parts of the world. That’s something I’m very proud of in this report: that we’re able to draw distinctions between how different people use it around the world and how those may make opportunities that much better, but also really heighten risks.
We also wanted to understand different language capabilities and how people were limited based on the languages they used in terms of what they could get from the LLMs. And lastly, I felt it was really important to blend the human rights approach with the computer science approach.
I come from many years in the business and human rights sector. It tends to be much more focused on the lawyers and policy and interactions with the engineers, but not the deep involvement. I thought it was important that we really draw that in, and you can see in our report we do specific evaluations of model evaluations and tie them to risks.
A complementary piece that Vyoma led, with our collaboration, is a model evaluation of political journalists using it in their work and tying those risks to human rights risks.
Vyoma Raman: I’ve heard time and time again from computer scientists that it’s important to care about rights and ethics and those sorts of things, but what I have the power and the expertise to do is to tune this parameter in the model that I end up training, and I don’t really understand how that relates to rights. And so part of what was really exciting to me about this work and why I found it super important is connecting the dots between groups that want similar things for the world and simply speak different languages and have different methods.
This report uses qualitative methodologies to understand the impact of technologies and our human rights evaluation framework, which uses standard evaluation metrics and combines them together under different interpretations so that computer scientists can think about ways of operationalizing rights in their methods. They approach the problems from adjacent angles.
And I think it’s really important to have both, first so that both sides can be convinced that there is a way to interact in a generative way and that both groups can work on the same problems, and you’re not foreclosed from one because of the methods that you have the ability to implement. But also so that we could start with the translation of one group’s methods to another, one group’s findings to another.
Gwyneth Shaw: The three of you interviewed people from the sectors of law, journalism, and education. What were the similarities and differences in the way folks in these fields are using or considering using generative AI?
Camille Chabot: Two things: First, the use of LLMs when we started the report was pretty widespread among the professionals we interviewed. And second, something else that was very striking was maybe the Global North-Global South difference that we observed.
That was, I think, a very important feature of this report. We collectively have different geographic expertise and I think collectively we speak six languages, so it was very important for us to integrate this analysis and this comparison of not only professionals in different fields, but also in different areas of the world.
That’s where one of the striking differences came about. We observed that there was more, or there tended to be more, risk aversion among the professionals that we interviewed in the Global North, who tended to emphasize the risks of large language models in terms of human rights. And we observed more enthusiasm in the Global South. We saw that a lot of professionals that we interviewed in Africa, in Asia, were very much looking at the opportunities, at the cost-saving opportunities, at the increased access to information, for example, when their counterparts in the Global North were talking primarily about the risks.
Vyoma Raman: When we were analyzing our data, I think one thing that we ended up discussing quite extensively was to what extent we want to categorize people from different geographies together. We understand the labels of Global North and Global South are not universally used, and also there have been many valid critiques of them. But what we were trying to get at here was both connecting this to the interest of translation — language that has been used in human rights spaces for a long time and that is easily recognizable there — but also very clearly showing that the communities where we are seeing excitement versus risk aversion relate to the opportunities that are available for the different sectors.
Betsy Popken: I’ll raise two things that we saw commonly across journalism, law, and education. One was the use of large language models to expedite tasks. Journalists in South Africa and India used it to write information about outcomes of sporting events, and significantly cut back the time to do that task.
Educators in Singapore used it to assess student work products, which presumably cost them less time to do so than to do it themselves free of LLMs. We also saw that legal professionals in Qatar use it to translate legal documents, which significantly cut back on the time used to do that. So that was a big thing we saw across all three professions, and we also saw cost savings come in as a motivating factor across all three.
In another example, the United Arab Emirates had an AI tutor which tutored hundreds of students, both those who were not native English speakers but also those with learning disabilities. And certainly it cost a lot less for the education system to use that AI tutor system rather than paying human tutors. In Singapore, they created a system to help complainants file documents at small claims court, and that was significant. That costs no money to the claimants, whereas otherwise they would have had to hire a legal professional to help them do so. So those are just a couple cross-cutting things we saw across all three of the different fields we studied.
Gwyneth Shaw: What are some of the risks that you discovered for each sector?
Camille Chabot: We tried to use a novel approach in that we analyzed both risks and opportunities at the same time because we saw that these risks and opportunities were actually connected to one another, and we did that across factors. We tried to measure the scope of each risk, the scale, and the remediability. With that in mind, we analyzed the most salient risks and opportunities for education, law, and journalism. For law, we found that the most salient risk was probably privacy, a risk that came about when lawyers, litigants, or judges essentially input confidential case information, including certain personal information about either plaintiffs, defendants, or other parties in the case.
Across the board, what most of the professionals we interviewed were most concerned about in terms of opportunity, and it’s an opportunity that we saw actually in other fields as well, is certainly access to information. Legal language can be very technical, very heavy, and so we saw that a lot of professionals and experts saw in LLMs an opportunity for legal professionals as well as litigants to understand and translate legal and other technical documents, therefore really potentially democratizing access to justice, or at least understanding of the legal system.
For education, we also saw access to information as the most salient opportunity, especially in the ability to access more complex and diverse information, including through translation of certain documents from other languages into English or from English to other languages, and creation of tailored explanation for students with special needs or who learn a little differently we saw in terms of salient risk, the risk to freedom of thought, which is potentially particularly concerning for young children when students over-rely on LLMs without critical thinking, or when educators rely on LLMs to generate feedback. That can really impede students’ ability to develop freedom of thought and use critical thinking.
For journalism, the most salient risk was freedom of thought as well. We saw that or we learned that a lot of journalists are very constrained by time, and so there can be this temptation to rely on LLMs to write, source, and fact-check articles, and that can really erode journalists’ capacity to select, verify, and analyze information. When it comes to opportunity, I think that access to information is a very salient one because LLMs have the ability of freeing up journalists’ time by doing repetitive tasks, such as summarization and editing, and that can free up time for journalists to focus on work where humans are needed, such as investigative work and in-depth reporting.
Betsy Popken: I’d like to underscore one thing in particular, and that is the freedom of thought. I think this was the biggest one that really stood out to us as one that would be significantly at risk across all three sectors.
That’s because LLMs enable you to do things without having to put all of the forethought into it, without having to organize your thoughts, without having to research. It cuts back on so many steps. I think one of the reasons Camille mentions it in the education sector is when you’re a young student and you are learning how to read, how to research, how to structure an outline, how to revise that outline, how to write that paper — those are all steps that one needs to learn in order to develop their own kind of freedom of thought and understanding of processes.
I think that was the biggest area that we saw at risk across sectors, but particularly those that could impact children whose minds are still developing.
I’d underline one more thing: We saw the same rights at risk and the same rights as opportunities. I mentioned that in the UAE there’s an education system using an AI tutor, that provides more tutor exposure to hundreds of more kids than would otherwise be able to access it. But at the same time, if that tutor is not aligned with the educational goals or the testing goal, it’s just not going to meet that right to education that the kids deserve. So it both is empowering the right to education and also potentially putting it at risk, depending on how well aligned the AI tutor is with what actually needs to be achieved.
Vyoma Raman: One part of our report was also reinterpreting research papers that have been produced by people interested in evaluating these technologies to understand when we put them in the context of these specific fields and these specific problems that we identified through our interview assessment. One of them found that language models have a hard time connecting curriculum goals to the specific problems and questions and understanding what it takes to actually achieve a particular goal as opposed to something that is relevant, but maybe not precisely the way that educators with years and years of experience would find them to be.
Gwyneth Shaw: You make a lot of recommendations in this report. Can you walk through some of what each of you think are the most important recommendations and some of the places where you think those recommendations might be listened to and applied?
Vyoma Raman: Language model evaluation is done very extensively by the companies that are releasing these tools as a way to show what they add over their competitors’ tools or the previous generation of tools that they have released themselves. They have a lot of resources, companies like OpenAI, to be able to conduct large scale evaluations that go far beyond even what we were able to do through our assessment.
So since many of the evaluations that they already do focus on the capabilities of these models in terms of doing math or writing or understanding things, we ask what it would take for them to extend those evaluations to focus concretely on human rights risks and opportunities.
There are multiple different actors through the supply chain of creating language models: There are developers who often create the models themselves. There are deployers that release them to the world and some sort of user-facing application. Many of the time, these two actors are the same thing.
But we also think that in terms of how to assess language models and human rights, they both offer various different opportunities. So developers who are very closely involved in the model have the opportunity to evaluate human rights risks upstream,
Whereas deployers, corporations that release models to the world, they’re very suited to do evaluations that focus on what the tasks are that people are really actually using these models for, and how can we understand and conceptualize the human rights risks and opportunities that appear in those particular contexts and develop evaluations around them? That one is a lot more context-specific, and something that I think that all of us have touched on already is the fact that we saw very different responses based on where the people we talked to were located and the level of resource access they had.
We also strongly recommend conducting international assessments specifically that attempt to understand these differences and evaluate the risks and opportunities with respect to them, because a particular use case with one set of resources could be a risk for some people and an opportunity for others, depending on sort of the resources and safeguards they have baked into the pipeline that they’re using.
Our last recommendation is about cultivating critical AI literacy. So here it becomes really important to think about what sorts of training do people have in doing the task and what the task actually ought to look like, what’s a good example of a particular task, and to understand how they use AI to support the work as opposed to replace it and never develop the taste or the cognitive abilities to perform the tasks themselves and to judge what is a good or bad version of it.
Critical AI literacy will become increasingly important as we move toward the future because we talked to professionals and experts who have already been doing what they’re doing for a very long time and have had their workflows altered in some significant way as a result of AI systems.
But as we see people growing up with access to language models, we won’t expect the skill development to look the same. So there is a question of how can we ensure that, for instance, young legal professionals get training in what documents are supposed to look like, how to put them together, like what all the nuances are in terms of things that can cause errors that maybe a language model might not know or notice?
Thinking towards the future, how do we design how the various professional skills are taught, and also how AI and the use of AI is taught alongside them, to not overshadow their ability to understand how the task should be done and to create a good output themselves while also receiving the benefits of lower cost and higher efficiency?
Betsy Popken: I think an important aspect of our recommendations, which was also a very challenging aspect of putting together these recommendations, was the six actors that we identified. That’s a lot of actors, and we had a lot of time identifying at what stage of development and deployment each of this actor, each of these actors was most competent to intervene and to decrease the chances of risks So just to take a quick example, when we looked into the regulation of LLMs and the setting guidelines, we saw that there are some risks that are common to many different domains, including law, education and journalism.
These call for more broad regulation, and governments should be in charge of that. But then when it comes to domain-specific risks, such as right to privacy in the legal field, for example, we found that other actors here, such as industry groups, probably have the expertise and need to work with professionals to create guidelines and protect users from risks.
We provide recommendations to six different stakeholder groups, starting with LLM developers and LLM deployers. Under LLM developers, we have resource providers and model engineers, and then under deployers, we have distributors, operators, industry groups, and government. Overall, we make nine recommendations. Some of them crosscut across all six stakeholder groups, and examples of those are our recommendation to assess human rights impacts of LLMs. We think all developers, all deployers, should be doing that, and they could do it at a large scale or small scale level. We believe all six stakeholder groups should be defining remediation stages where they can fix what has gone wrong in order to mitigate the risks. We also think all six of these stakeholder groups should be involved in cultivating critical AI literacy.
Some of our recommendations are specific to one or two of the stakeholder groups, and there are a number of them where we see a need for collaboration across the stakeholder groups that we list. We believe each of these six stakeholder groups may find different human rights impacts based on their assessment or how in-depth it was.
We think they should be talking, that each of these different stakeholder groups has something different to bring to that literacy space, and they should all be engaged with one another on that.
Gwyneth Shaw: As this technology continues to evolve, we’re seeing new issues all the time. What are some questions that might have come up for future exploration while you were working on this?
Betsy Popken: Our hope is to share what we have learned to as broad of an audience as possible across these six different stakeholder groups. We have two webinars coming up, one with the Global Network Initiative, which is a membership organization of technology companies, academics that focus on technology civil society organizations.
We also have one that will be hosted by Dropbox at some point in November across not just the Dropbox community, but others in that industry space. Most of our outreach and engagement so far has been with the LLM developer space and LLM deployer space.
The two parts that we’re really not tapping currently and we intend to expand into is the industry groups and government space. So that’s our next to-do with this particular project. If we were able to extend it out into other professional fields, I’d love to look at the use of it in the national security space, since we’re seeing so many relevant uses right now with really high risks associated with them.