In this episode, host Gwyneth Shaw talks with the authors of a recent report from Berkeley’s Human Rights Center, “An International Analysis of the Human Rights Impacts of Large Language Models” : Human Rights Center Executive Director Betsy Popken and Vyoma Raman and Camille Chabot, two former center researchers.
The report focuses on how the use of large language models, or LLMs — the foundation of generative AI — are affecting the fields of law, journalism, and education. Over 15 months, the team interviewed 56 people from 24 countries, including at least one expert and one practitioner in each sector on each inhabited continent.
The results are a window into what the benefits, and drawbacks, are for this technology in these sectors. And the authors conclude with a robust set of recommendations aimed at identifying and remediating potential harms in these important fields.
About:
“Berkeley Law Voices Carry” is a podcast hosted by Gwyneth Shaw about how the school’s faculty, students, and staff are making an impact — in California, across the country, and around the world — through pathbreaking scholarship, hands-on legal training, and advocacy.
Episode Transcript
[MUSIC PLAYING]
[GWYNETH SHAW]
Hi, listeners. I’m Gwyneth Shaw, and this is Berkeley Law Voices Carry, a podcast about how our faculty, students, and staff are making an impact through pathbreaking scholarship, hands-on legal training, and advocacy. In this episode, I’m joined by three great guests, the author of a recent report from Berkeley’s Human Rights Center, an international analysis of the human rights impacts of large language models.
They’re Human Rights Center Executive Director Betsy Popken and two former technology researchers for the center: Vyoma Raman, who’s now pursuing a PhD in information science at Cornell, and Camille Chabot, who just started her JD at Harvard Law School. The report focuses on how the use of large language models, or LLMs, the foundation of generative AI, are affecting the fields of law, journalism, and education. Over 15 months, the team interviewed 56 people from 24 countries, including at least one expert and one practitioner in each sector on each inhabited continent. The results are a window into what the benefits and drawbacks are for this technology in these sectors, and the authors conclude with a robust set of recommendations aimed at identifying and remediating potential harms in these important fields.
Thanks so much to all of you for joining me. How did the idea for this study come up? You’re building on some previous work, correct?
[BETSY POPKEN]
For me, the idea really came up when ChatGPT was launched to the public at the end of 2022. I began thinking not just what the advantages were, but also what the potential risks might be. And I would say at that time, so much of it was focused on the opportunities versus the harms that could come as a result
So I began thinking, what could we do as the Human Rights Center to really make an impact in helping people understand the risks and providing recommendations for stakeholders to mitigate those risks and maximize the opportunities? So I began thinking about that at the end of 2022 and into 2023. I then was connected with Vyoma and Camille because they were former Investigations Lab students of the Human Rights Center, and they graduated that year in 2023.
And one of our faculty directors, Alexa Koenig, connected me up with both of them. We sat together and we decided how we would progress, looking at both the risks and then really wanting to incorporate Vyoma’s expertise as a computer scientist as well. My background in this came from when I worked at a law firm, Orrick, Herrington Sutcliffe. I co-founded and led the business and human rights practice at the firm, and many of our clients there were technology company clients.
And together we worked with those clients to understand the risks of their technology products and services and develop plans, both policies and procedures to help mitigate those risks. So that was the background through which I came to this project and working on this.
[CAMILLE CHABOT]
I had been thinking about the impact of new technologies on human rights for a few years working at the Human Rights Center, both in terms of investigations but also in terms of new harm, such as the use of drones, for example.
And so when ChatGPT came out, I remember thinking to myself that this new technology had both tremendous impact, but also a lot of risks associated with its use, especially in professional fields. And I was very attracted to the idea of conducting this kind of research in an interdisciplinary setting, which I had already experienced at the Human Rights Center, and the opportunity to collaborate with both Betsy, who had a lot of experience at a private firm, and with Vyoma, a computer scientist, and thinking through these questions alongside lawyers, experts, and professionals was really attractive to me
[VYOMA RAMAN]
I was likewise very drawn to the opportunity to connect with Betsy and Camille about this topic because coming from more of the computer science side, one of the gaps that I felt that I had really seen is an understanding of How these tools are creating real grounded impact in different communities as opposed to simply being something that one puts out into the world and sees what happens next. And I think this is something that I find to be maybe a recurring problem or question in computer science spaces is, how do we pick the problems we choose to solve? Whose problems matter?
And why are we choosing to formulate problems in one way or another? When you’re building an AI system, you are, especially with these modern forms of AI, optimizing it on a particular objective or goal based on a whole lot of data that is supposed to represent the world in some way But there are challenges with both which data do you use and who is represented and who is not represented, and also how exactly do you formulate the objective and the goal.
And I think this has been studied quite a lot in sort of the algorithmic fairness literature at the level of the technical systems. And I saw this project as an opportunity to go beyond that, to connect to the downstream human rights impact in a way that human computer interaction scholars end up doing. And I think working with Betsy and Camille was especially valuable because of their expertise in law and international relations that helped to ground exactly what the problems Could be and design our methodology around excavating those because I think the design of the methodology can also drastically influence what you end up finding in these situations.
[GWYNETH SHAW]
Why was it important to study and analyze what’s happening given the perspectives that all of you were coming from? What really drove you to take this on?
[BETSY POPKEN]
For me, it was that so little attention was being paid to the risks. I think early on we saw analyses happening in the medical field because that was one where people could actually see the tangible risks of something going wrong, but we didn’t see it in any other professional field. And I have a background in law, and I’m an educator.
We also have a number of journalists on our team at the Human Rights Center, so those were three fields where we felt like we could really begin to understand what those risks might be, and we’d have kind of connections to people all around the world We were also interested as a team in understanding the different risks based upon how people use it in different parts of the world. And so that was something I’m very proud about this report, that we’re able to draw distinctions between how different people use it around the world and how those may make opportunities that much better, but also really heighten risks And we also wanted to understand different language capabilities and how people were limited based on the languages they used in terms of kind of what they could get from the LLMs. And just lastly, I felt it was really important to blend the human rights with the computer science approach.
I come from many years in the business and human rights sector. It tends to be much more focused on kind of the lawyers and policy with kind of interactions with the engineers, but not the deep involvement. And so I thought it was important that we really draw that in, and you can see in our report we do specific evaluations of model evaluations and tie them to risks.
And a complementary piece we did that Vyoma can speak more about is she led, with our collaboration, a model evaluation of political journalists using it in kind of their work and tying those risks to human rights risks. So I’ll let Vyoma speak more about that, but those are the three things I just thought very little attention paid to risks. We needed to focus on geography and language, and we needed to pair the human rights with the computer science
[VYOMA RAMAN]
Yeah, and I can maybe give a little bit more context as well. I think something that’s or a narrative that I’ve heard time and time again from computer scientists is I think it’s important to care about rights and ethics and those sorts of things, but what I have the power and the expertise to do is to tune this parameter in the model that I end up training, and I don’t really understand how that relates to rights. And so part of what was really exciting to me about this work and why I found it super important is connecting the dots between groups that want similar things for the world and simply speak different languages and have different methods.
The two Reports that we ended up releasing, this one that is an international analysis goes in one direction. It uses qualitative methodologies to understand the impact of technologies and our human rights evaluation framework, which uses standard evaluation metrics and combines them together under different interpretations so that computer scientists can Think about ways of operationalizing rights in their methods. They approach the problems from adjacent angles.
And I think it’s really important to have both, just first, so that both sides can be convinced that there is a way to interact in a generative way and that both groups can work on the same problems, and you’re not foreclosed from one because of the methods that you have the ability to implement. But also, so that we could start with the translation of one group’s methods to another, one group’s findings to another. So for the rights evaluation paper which was published through the AI Ethics and Society Conference run by the American—or the Association for the Advancement of Artificial Intelligence.
That one was about, like, how can we take the language from human rights instruments like the UNGPs and really Operationalize that into metrics that computer scientists implement every day as they are building new systems, and how can we then attempt to give a score to notions of risk saliency like scope, scale, and likelihood that computer scientists can provide the score and human rights practitioners can take that and interpret it in the contexts that they are working in
[GWYNETH SHAW]
I’m so glad you talked about that specifically because I think that’s a really important connection, and it’s great to hear that perspective. The three of you interviewed people from the sectors of law, journalism, education, as you mentioned, Betsy. What were the similarities and differences in the way folks in these fields are using or considering using generative AI?
And I know there’s a lot of variation not just among the sectors, but among the places where people are located
[CAMILLE CHABOT]
So I think Betsy will address a little bit more in details of how folks in each of these fields used LLMs, but I think two things. The first is that the use of LLM when we started the report was pretty widespread among the professionals we interviewed. And something else that was very striking was maybe the global north, global south difference that we observed.
So I want to start by saying that was, I think, a very important feature of this report. We collectively have different geographic expertise. I think we collectively speak six languages, and so it was very important for us to integrate this analysis and this comparison of not only professionals in different fields, but also in different areas of the world.
And I think that’s where one of the striking differences came about. We observed that there was more, or there tended to be more risk aversion among the professionals that we interviewed in the Global North, who tended to emphasize the risks of large language models in terms of human rights. And we observed more enthusiasm in the global south. We saw that a lot of professionals that we interviewed in Africa, in Asia, were very much looking at the opportunities, at the cost-saving opportunities, at the increased access to information, for example, when their counterparts in the global north were talking primarily about the risks.
So I thought that was a very interesting feature.
[VYOMA RAMAN]
Can I just jump in real quick here? When we were analyzing our data, I think one thing that we ended up discussing quite extensively was to what extent we want to Categorize people from different geographies together. We understand the labels of Global North and Global South are not universally used, and also there have been many valid critiques of them.
But I think what we were trying to get at here was both connecting this to the interest of translation, language that has been used in human rights spaces for a long time and that is easily recognizable there. But also very clearly showing that The communities where we are seeing excitement versus risk aversion relate to the opportunities that are available for the different sectors that Betsy will talk about more and the extent to which there are other resources available compared to language models to fill a gap
[BETSY POPKEN]
I’ll just raise two things that we saw commonly across journalism, law, and education. One was the use of large language models to expedite tasks, and just a couple of examples are journalists in South Africa and India used it to write information about outcomes of sporting events, right? So that significantly cut back time to do that task.
Educators in Singapore used it to assess student work product, which presumably cost them less time to do so than to do it themselves free of LLMs. We also saw that legal professionals in Qatar use it to translate legal documents, which significantly cut back on kind of the time used to do that. So that was a big thing we saw across all three professions. I’d say we also saw cost savings come in as a motivating factor across all three.
And just as a couple examples, the UAE had an AI tutor which tutored hundreds of students, both those who were not native English speakers, to translate into kind of the native Arabic language that was used, but also those with learning disabilities. And that was certainly it cost a lot less for the education system to use that AI tutor system rather than paying human tutors. Another example is in Singapore.
They created a system to help complainants file documents at small claims court, and that was significant. That costs no money to the claimants, whereas otherwise they would have had to hire a legal professional to help them do so. So those are just a couple cross-cutting things we saw across all three of the different fields we studied
[GWYNETH SHAW]
What are some of the risks that you discovered for each sector? In addition to some of the benefits you just talked about, saving time, saving money, I think even those of us who don’t use a lot of LLMs can see that. But what are some of the risks that you discovered in these interviews?
[CAMILLE CHABOT]
I want to start by saying that we tried to use a novel approach in that we analyzed both risks and opportunities at the same time because we saw that these risks and opportunities were actually connected to one another, and we did that across factors. We tried to measure the scope of each risk, the scale, and the remediability. And with that in mind, we analyzed the most salient risks and opportunities for education, law, and journalism for law, we found that the most salient risk was probably privacy, a risk that came about when lawyers, litigants, or judges essentially inputted confidential case information, including certain personal information about either plaintiffs, defendants, or other parties in the case.
And that was across the board what most of the professionals we interviewed were most concerned about in terms of opportunity, and it’s an opportunity that we saw actually in other fields as well, is certainly access to information. Legal language can be very technical, very heavy, and so we saw that a lot of professionals and experts saw in LLMs an opportunity for legal professionals as well as litigants to understand and translate legal and other technical documents, therefore really potentially democratizing access to justice, or at least understanding of the legal system For education, we also saw access to information as the most salient opportunity, especially in the ability to access more complex and diverse information, including through translation of certain documents from other languages into English or from English to other languages, and creation of tailored explanation for students with special needs or who learn a little differently we saw in terms of salient risk, the risk to freedom of thought, which is potentially particularly concerning for young children when students over-rely on LLMs without critical thinking, or when educators rely on LLMs to generate feedback. And so that can really impede students’ ability to develop freedom of thought and use critical thinking. So for journalism, the most salient risk was freedom of thought as well.
We saw that or we learned that a lot of journalists are very constrained by time, and so there can be this temptation to rely on LLMs to write, source, and fact-check articles, and that can really erode journalists’ capacity to select, verify, and analyze information. When it comes to opportunity, I think that access to information is a very salient one because LLMs have the ability of freeing up journalists’ time by doing repetitive tasks, such as summarization and editing, and that can free up time for journalists to focus on work where humans are needed, such as investigative work and in-depth reporting.
[BETSY POPKEN] I’d like to underscore one thing in particular, and that is the freedom of thought. I think this was the biggest one that really stood out to us as one that would be significantly at risk across all three sectors.
And that’s because LLMs enable you to do things without having to put all of the forethought into it, without having to organize your thoughts, without having to research, without, it cuts back on so many steps, and I think one of the reasons Camille mentions it in the education sector is this is when, when you’re a young student and you are learning how to Read, how to research, how to structure an outline, how to revise that outline, how to write that paper. Those are all steps that one needs to learn in order to develop their own kind of Freedom of thought and understanding of processes. And so I think that was the biggest area that we saw at risk across sectors, but particularly those that could impact children whose minds are still developing.
I’d underline one more thing that has been raised already, but that’s that we saw the same rights at risk and the same rights as opportunities. Maybe I’ll bring up something I raised previously, but I mentioned in the UAE there’s an education system using an AI tutor, right? That provides more tutor exposure to hundreds of more kids than would otherwise be able to access it.
But at the same time, if that tutor is not aligned with the educational goals or the testing goal, it’s just not going to meet that right to education that the kids deserve. So it both is empowering the right to education- But it’s also potentially putting it at risk, depending on how well aligned the AI tutor is with what actually needs to be achieved.
[VYOMA RAMAN]
And one part of our report was also reinterpreting research papers that have been produced by people interested in evaluating these technologies to understand when we put them in the context of these specific fields and these specific problems that we identified through our interview assessment. Like, what does the research say about whether those problems are what we call real or whether they’re actually issues. And one of the papers that we reinterpreted that is relevant to what Betsy was just saying is that language models have a hard time connecting curriculum goals to the specific problems and questions and understanding what it takes to actually achieve a particular goal as opposed to something that is relevant, but maybe not precisely the way that educators with years and years of experience would Find them to be,
[GWYNETH SHAW]
I think, that’s a really interesting connection to make what an LLM can do and can’t do in that regard. You make a lot of recommendations in this report, and I’m not going to detail all of them because I want our listeners, if they’re interested, to go and read the whole report. There’s a link in the show notes for sure.
Can you walk through some of what each of you think are the most important recommendations and some of the places where you think those recommendations might be listened to and applied?
[VYOMA RAMAN]
Yeah, absolutely. So I will go through a couple, and Camille and Betsy, feel free to add on as you would like. So the first that we have listed in our report is assessing the human rights impacts of large language models So language model evaluation is done very extensively by the companies that are releasing these tools as a way to show what they add over their competitors’ tools or the previous generation of tools that they have released themselves They have a lot of resources, oftentimes companies like OpenAI, to be able to conduct large scale evaluations that go far beyond even what we were able to do through our assessment.
So since many of the evaluations that they already do focus on the capabilities of these models in terms of doing math or writing or understanding things, that sort of thing, like we ask What it would take for them to extend those evaluations to focus concretely on human rights risks and opportunities. And in many ways, both of the reports that we release show different ways of approaching the same goal of evaluating with respect to human rights. And To be even more specific, there are multiple different actors through the supply chain of creating language models.
There are developers who often create the models themselves. There are deployers that release them to the world and some sort of user-facing application. Many of the time, these two actors are the same thing.
But we also think that in terms of how to assess language models and human rights, they both offer various different opportunities. So developers who are very closely involved in the model have the opportunity to evaluate human rights risks upstream, which is not really what we did in our evaluation, looking at the data that is being used, the data labeling and collection practices, and sort of risk in that realm. As well as if you’re looking at, for instance, demographic biases in the outputs of language models, they’re also able to evaluate at that more granular model output level
Whereas deployers, people who release or corporations that release models to the world, they’re very suited to do evaluations that focus on, okay, what are the tasks that people are really actually using these models for? And how can we understand, conceptualize the human rights risks and opportunities that appear in those particular contexts and develop evaluations around them? And so that one is a lot more context specific And something that I think that all of us have touched on already is the fact that we saw very different responses based on where the people we talked to were located and the level of resource access they had
So we also strongly recommend conducting international assessments specifically that attempt to understand these differences and evaluate the risks and opportunities with respect to them, because a particular use case with one set of resources, as Betsy mentioned, could be a risk for some people and an opportunity for others, depending on sort of the resources and safeguards they have baked into the pipeline that they’re using. The last thing I will touch on is our last recommendation, which is cultivating critical AI literacy. And that means like for instance, in the context of the children that Betsy mentioned, who may experience risks to freedom of thought if they don’t develop their cognitive abilities as a result of over-reliance on AI systems.
So here it becomes really important to think about What sorts of training do people have in doing the task and what the task actually ought to look like? What is a good example of a particular task? And understand how they use AI to support the work as opposed to replace it and never develop the taste or the cognitive abilities to perform the tasks themselves and to judge what is a good or bad version of it And critical AI literacy, I think, will become increasingly important as we move toward the future because we talked to professionals and experts who have already been doing what they’re doing for a very long time and have had their workflows altered in some significant way as a result of AI systems.
But as we see people growing up with access to language models, we won’t expect the skill development to look the same. So there is a question of- How should we ensure that, for instance, young legal professionals get training in what documents are supposed to look like, how to put them together, like what all the nuances are in terms of things that can cause errors that maybe a language model might not know or notice?
Thinking towards the future, long term, how do we design the way that various professional skills are trained or Scratch that, how the various professional skills are taught, and also how AI and the use of AI is taught alongside them to not overshadow their ability to understand how the task should be done and to create a good output themselves while also receiving the benefits of, as Betsy mentioned, lower cost, higher efficiency, those sorts of things
[CAMILLE CHABOT]
I think an important aspect of our recommendations, which was also a very challenging aspect of putting together these recommendations, is as Vyoma highlighted, the six actors that we identified. That’s a lot of actors, and we had a lot of time identifying at what stage of development and deployment each of this actor, each of these actors was most competent to intervene and to decrease the chances of risks So just to take a quick example, when we looked into the regulation of LLMs and the setting guidelines, we saw that, as Betsy already mentioned, there are some risks that are common to many different domains, including law, education and journalism. And these call for more broad regulation.
And for example, governments should be in charge of that. But then when it comes to domain-specific risks, such as right to privacy in the legal field, for example, we found that other actors here, industry groups probably have the expertise and need to work with professionals to create guidelines and protect users from risks. Maybe I’ll start from a high-level perspective.
So we provide recommendations to six different stakeholder groups, LLM developers and LLM deployers. Under LLM developers, we have one, resource providers, two, model engineers. And then under deployers, we have three distributors, four operators, five industry groups, and six government.
So we provide recommendations to these six stakeholder groups. Overall, we make nine recommendations. Some of them crosscut across all six stakeholder groups, and examples of those are our recommendation to assess human rights impacts of LLMs. We think all developers, all deployers should be doing that, and they could do it at a large scale or small scale level.
We believe all six stakeholder groups should be defining remediation stages where they can fix what has gone wrong in order to mitigate the risks. We also think all six of these stakeholder groups should be involved in cultivating critical AI literacy. Some of our recommendations are specific to one or two of the stakeholder groups, and that is one of the things Camille mentioned, regulating general purpose AI.
That should be government policymakers. Another one is setting guidelines for the use of LLMs in a professional context. That should be operators such as a law firm, and industry groups such as the International Bar Association.
Those should be the ones setting guidelines for the use of LLMs in professional contexts. Another example is creating user feedback mechanisms. That should be both the distributors and the operators.
And then a fourth one that I’ll mention is designing LLM interfaces responsibly. That should be distributors. So we have identified not just Nine different recommendations, but we’ve assigned them to the relevant stakeholder group, one of six.
And there are a number of them where we see a need for collaboration across the stakeholder groups that we list. And some of these have already been raised. Yeoman talked about assessing human rights impacts. We believe each of these six stakeholder groups may find different human rights impacts based on their assessment or how in-depth it was.
We think they should be talking. We think tailoring tools to professional tasks, we think that operators and industry groups should be talking to one another about that. Designing LLM interfaces responsibly.
We think that, or sorry, scratch that Cultivating critical AI literacy, we think that each of these different stakeholder groups has something different to bring to that literacy space, and they should all be engaged with one another on that. So I guess I would say we have the six stakeholder groups, nine recommendations, and while not all recommendations are for each stakeholder group, we think they should be collaborating across a number of the recommendations.
[GWYNETH SHAW]
As this technology continues to evolve, we’re seeing new issues all the time. What are some questions that might have come up for future exploration while you were working on this?
[BETSY POPKEN]
So for this report in particular, we have two webinars coming up.
Our hope is to share what we have learned to as broad of an audience as possible across these six different stakeholder groups. We have two webinars coming up, one with the Global Network Initiative, which is a membership organization of technology companies, academics that focus on technology Civil society organizations. And so that will be coming up at some point in October with their membership base.
We also have one that will be hosted by Dropbox at some point in November across not just the Dropbox community, but others in kind of that particular industry space. In addition to that, when we released this report, we had a piece in Tech Policy Press, which again speaks to more of kind of the tech policy audience, and we expect to have a podcast there as well. So most of our output thus far and engagement will be with folks in some of the initial LLM developer space and LLM deployer space.
The two parts that we’re really not tapping currently and we intend to expand into is the industry groups and government space. So that is our next To do on this particular project. If we were able to extend it out into other professional fields, I’d love to look at kind of the use of it in national security space since we’re seeing so many relevant uses right now with really high risks associated with them.
[VYOMA RAMAN]
As far as my work, I’m currently doing my PhD at Cornell Tech, where I am studying language technologies broadly and how their impacts can have different kinds of harms in the world and how to also Think more critically about how we might mitigate those harms at the level of working with people who end up building the models, model engineers among our six categories of actors. So some of my work lately has looked at the implications of different design decisions for the outcomes of tasks related to language modeling. And I think this is going to be a very widespread theme going forward because this work, I think, answered at least partially the question of which problems matter, how are people using it, what should people who create these tools focus on?
And now I am trying to understand, okay, we have some of those answers now. How can we develop the technical methodologies to put those into practice?
[CAMILLE CHABOT]
And I am currently in my first year of law school here at Harvard, and I think I want to bring the understanding that I developed through this report into my career as a lawyer. Specifically, I’d wanna zoom in on the access to justice questions and how it can be used responsibly to really enhance the opportunities that AI represents for low income litigants and people who, for other reasons, do not have access to the legal system, while preventing the risks that are associated with hallucinations and other, and other technical issues that we have highlighted in the report
[GWYNETH SHAW]
Thank you all for joining me for this very timely conversation, and thank you, listeners. To learn more about my guests and their research, please check out the show notes where you’ll find a link to the report and to bios for each of them. And if you enjoyed this episode, please share it and be sure to subscribe to Voices Carry wherever you get your podcasts.
Until next time, I’m Gwyneth Shaw.
[MUSIC PLAYING]