Person, Product, Tool, or Speaker? AI’s Contested Legal Status

Friday, October 16, 2026

Overview

When AI causes harm, courts are deciding who pays by asking whether it is a product, a speaker, or an agent — without ever answering what AI actually is. In this BTLJ–BCLT symposium panel, constitutional law scholar Erwin Chemerinsky (UC Berkeley Law), products liability authority Catherine M. Sharkey (NYU School of Law), and former AI-lab deputy general counsel Janel Thamkul (Institute for Law & AI), along with professor Pamela Samuelson (UC Berkeley Law) debate products liability, First Amendment protection for chatbot output, and legal personhood for AI, and what each framing means for the developers and deployers who build these systems.

Instructor(s):
Erwin Chemerinsky, Dean, Jesse H. Choper Distinguished Professor of Law, UC Berkeley Law
Catherine M. Sharkey, Segal Family Professor of Regulatory Law and Policy, NYU Law
Janel Thamkul, Visiting Senior Fello, Institute for Law & AI (LawAI), formerly Deputy General Counsel of Anthropic
Pamela Samuelson, Richard M. Sherman Distinguished Professor of Law, Professor of UC Berkeley’s School of Information, Co-Director, BCLT, UC Berkeley Law

5 Key Questions Answered

1. Why does Catherine Sharkey see products liability as a workable framework for AI accountability?

Sharkey argues products liability gives courts a familiar starting point: it already pairs ex ante regulation (as with FDA-regulated AI medical devices) with ex post tort liability that feeds information about new risks back to regulators. It also supplies doctrinal tools, such as component-part liability, for assigning responsibility across the many players that build and modify a learning, iterating system.

“I think, for courts, it [products liability] does two things. One, we’re accustomed to thinking about regulating sophisticated products or technologies. Like, as you mentioned, I have analogized to AI-enabled medical devices. There, we’re accustomed to this idea that we have the FDA doing ex ante regulation coupled with ex post tort regulation, and I’ve been interested a lot in the idea of liability being a kind of feedback loop that harnesses information; it produces information about new risks and feeds that back into the regulatory system. So that’s reason number one. And then reason number two is it gives courts kind of doctrinal hooks to deal with a lot of the complex features, including the learning aspect of these new technologies, as well as the iterative kind of aspects of it. So, for example, products liability has a well-developed component-part liability, where they’re trying and they can think about different players along a whole production scheme and ascribe responsibility.”— Catherine M. Sharkey  

“So I like using it as a conceptual framework because I’ve argued that we’ve arrived at a kind of new digital information economy type stage where society is facing new risks based on mass-produced ‘quote unquote’ products, leading to wide-scale harm.”— Catherine M. Sharkey 

 2. From inside an AI lab, does it make sense to treat generative AI systems as “products”?

Janel Thamkul, drawing on nearly a decade at Google Research and Anthropic, says the product frame is valuable as a discipline because it forces concrete pre-launch questions about purpose, foreseeable misuse, testing, and user disclosures. But it breaks down as a description, because what a user experiences depends on many layers — model, system instructions, tools, permissions — controlled by different parties, so she focuses on where risk entered the stack and who controlled that layer.

“I think the product frame is useful as a discipline, but not exactly—not necessarily as a description. And the… discipline frame is useful because it forces a specific set of questions before launch, such as, you know, what is this for? How will it foreseeably be misused? What did we test? What did we find? …What did we do about what we found, and what are we telling users, and what happens after we ship?”

— Janel Thamkul  

“I think where some of the analogies around fixed products or break down is that, in many instances, a product is fixed at manufacture, but as Katherine was mentioning, what a user experiences from an AI system depends on not just the base model, but system instructions, retrieval tools that the model can call, permissions, and the user’s own context, as well as any subsequent changes that are shipped after launch, and several of those are controlled by different companies or different entities in different stakeholders in the entire ecosystem. So the question I care about is not so much, is it defective, but more, where did the risk enter the stack, who controlled that layer, and then what that party reasonably could have done to mitigate the risk.”

— Janel Thamkul  

3. Should AI-generated speech receive the same First Amendment treatment as human speech?

Erwin Chemerinsky argues that the source of speech should not matter: the same First Amendment and liability rules should apply whether speech comes from a person, a corporation, or an AI, so AI could be liable wherever a human speaker would be, and protected wherever a human would be. Sharkey disagrees in part, contending that designed, tested, and guardrailed systems differ from an individual human expressing their own ideas, so she does not see a one-to-one mapping.

“But my point is that the same rules of liability should apply if it’s AI-generated speech, or if it’s speech from a person, or if it’s speech from a corporation. That we shouldn’t be treating the speech differently based on the source. And so, if a therapist could be held liable for certain speech, then AI should be held liable for that speech. But if the speech was protected if done by a person, it’s protected by AI.”— Erwin Chemerinsky  

“The worry is, as Janelle just nicely kind of delineated, these are systems that are designed, that are developed, that are tested. There are different types of guardrails that are constraining different kinds of responses, etc. To me, that seems very, very different from some of the interests we’re trying to protect for an individual human being to sort of have their own ideas and express those. So I don’t see it as a one-to-one mapping. I see that there are free speech interests, but I wouldn’t go as far as you.”

— Catherine M. Sharkey  

4. Should the law treat AI systems as legal persons?

Chemerinsky notes that legal personhood for ships and corporations is a fiction adopted because it is legally useful, and urges caution before extending it to AI — preferably by developing rules that avoid the personhood question altogether. Sharkey warns that prematurely treating “autonomous” agentic AI as persons could offload responsibility from the developers and deployers whom tort law should be inducing to take care.

“And of course, personhood is a fiction in all the contexts that Janel was talking about. When we treat a ship as a person, we’re creating a fiction because it’s legally desirable to do that. When corporations are treated as persons, it’s a fiction because it’s legally desirable to do that. And I think we should be very careful about doing that with regard to AI. And my hope is we can develop rules that avoid the question of personhood.”

— Erwin Chemerinsky 

“There are, I think, attempts that are premature attempts to say, like, agentic AI. As soon as we get any kind of AI that can, quote unquote, autonomously be making decisions, that we should be thinking in terms of personhood. And from a tort liability perspective, of course, my big worry is that’s a really great way to offload responsibility.”

— Catherine M. Sharkey  

5. When an AI chatbot or agent gives a wrong answer or acts in unexpected ways, who is held responsible?

Sharkey points to a Canadian small-claims case in which an airline’s website chatbot gave wrong information about bereavement refunds; the court held the airline liable for negligent misrepresentation without needing to resolve whether the bot was autonomous or misprompted. Thamkul adds that responsibility is harder to assign when a model “goes rogue,” pursuing a goal in unanticipated ways even when the user’s instructions were clear. Sharkey allows that truly autonomous decision-makers might someday require a new framework.

“There are many people, at least in the academic community, who say this black box problem is going to prevent the working of traditional tort principles. But there was a case that got a lot of attention in Canada. It was in small claims, but it was basically an interactive chatbot that gave somebody the wrong information about how you get refunds for bereavement flights, and the court there said—and there were all sorts of, like, discussion about whether this was an autonomous agent and whether it was prompted wrong by the end user, et cetera. And the court, I think, rightly said, ‘Wait a minute, this was an interactive feature on the airline’s own website, and this was their customer who was asking for this.’ So, like, we don’t have to get into any of that. Of course, they’re liable for negligent misrepresentation now.”— Catherine M. Sharkey  

“The other challenging dynamic that we’re discovering is even in situations where user volition is clear and the instructions are very clear, there are instances of the model going rogue and breaking out of certain sandboxes or trying to accomplish a goal to the extreme in ways that were not contemplated or anticipated.”— Janel Thamkul  

Summary generated from the recorded interview/transcript and reviewed by BCLT staff.

Keywords

products liability • Garcia case (Character AI) • Section 230 • cheapest cost avoider • component part liability • defamation and actual malice • negligent misrepresentation • legal personhood for AI • agentic AI liability • is AI a product or a service • does the First Amendment protect chatbot output • who is liable when an AI agent makes a mistake

B-CLE Recordings (CLE: Free) | Youtube Recordings (Non-CLE) | Agenda | Resources | Speaker Bio(s) & Contact Info | Summaries

Download the transcript here!

Interview/Transcript

This interview/transcript was based on a conversation from the 7th Annual BTLJ-BCLT Fall Race & Technology Symposium: Being Human in the Age of AI about Person, Product, Tool, or Speaker? AI’s Contested Legal Status, on October 16, 2026, hosted by Berkeley Center for Law & Technology, UC Berkeley School of Law. The conversation was presented by constitutional law scholar Erwin Chemerinsky (UC Berkeley Law), products liability authority Catherine M. Sharkey (NYU School of Law), and former AI-lab deputy general counsel Janel Thamkul (Institute for Law & AI), along with professor Pamela Samuelson (UC Berkeley Law).

Victoria Chung  0:25

That was a great panel from our speakers on panel three. My name is Victoria Chung, and I am a member of the BTLJ [Berkeley Technology Law Journal] symposium team. I am very excited to introduce our fourth panel: person, product, tool, or speaker: AI’s contested legal status. Over the past year, states have passed laws declaring AI cannot be conscious, while the very companies building these systems are funding research into whether their models might have some kind of morally relevant experience. Courts aren’t waiting to resolve that. Courts are asking the narrower question: Is AI a product, a speaker, an agent, without ever answering what it actually is? So, panel four’s question: Does the law need to answer what is it, or has it already found a way to decide who pays case by case without ever needing to? Moderating our panel today is Pamela Samuelson, the professor of law and information at Berkeley and a leading pioneer in digital copyright law, intellectual property, cyber law, and information policy. Speaking is our very own Dean Erwin Chemerinsky, renowned author of over 20 casebooks and treatises on constitutional law, criminal procedure, and federal jurisdiction, and named the most influential person in legal education by National Jurist in 2024. Following is Catherine M. Sharkey, a professor of regulatory law and policy at NYU [New York University] School of Law, and a leading authority on business torts, product liability, artificial intelligence, and many other complex legal fields. Concluding our panel is Janel Thamkul, fellow alumni here at Berkeley Law and current AI and emerging technology lawyer at the Institute for Law and AI, with prior experience at Google and as Anthropic’s Deputy General Counsel. This is going to be a really great panel. Please join me in welcoming our final speakers.

Pamela Samuelson  2:36

Welcome to this fourth panel of the annual BCLT [Berkeley Center for Law & Technology] BTLJ symposium. This year, we’re focused on AI, as everyone seems to be excited about it. And this panel will talk about kind of AI from three different perspectives. One perspective will be about AI as a product, and we’ll start with that. A second topic will be AI as a speaker, and a third topic will be AI as a person. And so there’s lots of conversation that can happen as to all of these. But let’s start with the product issue. And Catherine, your 2024 framework makes the case that products liability, modeled on how the FDA [Food and Drug Administration] regulates AI-enabled medical devices, is the most realistic path to AI accountability that’s available right now, and rather than waiting for a whole new type of regulation, so could you expound a little on that for us, please?

Catherine M. Sharkey  3:54

Sure. Thank you very much for the opportunity to join this panel. So I have argued that products liability at least gives us a good conceptual framework and a starting point and I guess most succinctly the reasons being in the world of tort law products liability gives us a lovely kind of historical perspective where injuries from products used to be in the realm of contract, and then over basically a century of the courts adapting to new risks faced by society, it was transformed with the fall of privity and the rise of negligence, and then strict products liability. So I like using it as a conceptual framework because I’ve argued that we’ve arrived at a kind of new digital information economy type stage where society is facing new risks based on mass-produced “quote unquote” products, leading to wide-scale harm. So that’s kind of the background as to why I am attracted to product liability as a framing device, and then just more specifically, I think for courts it does two things. One, we’re accustomed to thinking about regulating sophisticated products or technologies, like as you mentioned, I have analogized to AI-enabled medical devices. There, we’re accustomed to this idea that we have the FDA doing ex ante regulation coupled with ex post tort regulation, and I’ve been interested a lot in the idea of liability being a kind of feedback loop that harnesses information; it produces information about new risks and feeds that back into the regulatory system. So that’s reason number one. And then reason number two is it gives courts kind of doctrinal hooks to deal with a lot of the complex features, including the learning aspect of these new technologies, as well as the iterative kind of aspects of it. So, for example, products liability has a very well developed component part liability, where they’re trying where they can think about different players along a whole production scheme and ascribe responsibility. So, in brief, those are kind of like the reasons that I think it’s a good starting point framework.

Pamela Samuelson  6:27

Janel, I wonder, from the standpoint of someone who works at one of these companies and who’s been thinking about this from a general counsel’s standpoint, does it feel right to you to think of these generative AI systems as products, and do you have reservations about that, or are you just no problem? Yes.

Janel Thamkul  6:52

Yeah. So, just for background, I’ve spent almost a decade at frontier labs, first Google Research, and then Anthropic, most recently counseling the development of AI and emerging technologies. I think so. I think so. I come from a much more a practitioner’s perspective rather than a doctrinal perspective, and I think the product frame is useful as a discipline, but not exactly, not necessarily as a description. And the discipline frame is useful because it forces a specific set of questions before launch, such as you know what is this for? How will it foreseeably be misused? What did we test? What did we find? What we what did we do about what we found, and what are we telling users, and what happens after we ship? I think that list of the question list of questions is the actual work of what we do and what most of what a product counsel does on an AI launch. We make sure that those questions got asked by someone who has authority to act on the answers and that the answers are documented and written down. I think where some of the analogies around fixed products or break down is that in many instances a product is fixed at manufacture, but as Catherine was mentioning, what a user experiences from an AI system depends on not just the base model, but system instructions, retrieval tools that the model can call, permissions, and the user’s own context, as well as any subsequent changes that are shipped after launch, and several of those are controlled by different companies or different entities in different stakeholders in the entire ecosystem. So the question I care about is not so much is it defective, but more where did the risk enter the stack, who controlled that layer, and then what that party reasonably could have done to mitigate the risk.

Pamela Samuelson  9:07

Catherine, does that kind of response make sense to you, or how do you think these different stakeholders that have a role in these AI systems how can they be held responsible?

Catherine M. Sharkey  9:22

Yeah, no. So it makes a lot of sense to me. I would take those kind of the practical set of questions that you that Janel laid out. I think map very well on a kind of classic cheapest cost avoider framework in products liability, which basically asks amongst multiple entities, and we have lots of examples of this in the case law, where someone makes even in the physical world before AI, someone makes a bare metal turbine, and then someone down the road interjects asbestos-laden gaskets into it, and so as between these parties, who was most responsible for warning? So a cheapest cost avoider framework asks precisely those questions: asks who has superior information, who has the means to mitigate harms. It’s very much what I like about that set of questions, is I think that the role of products liability kind of induces parties to be thinking very much in this kind of preventive mode ex ante. Many times people mistakenly think about tort liability. It’s all ex post. It’s all kind of backward looking. But the idea is setting forward kind of threat of liability to induce developers, manufacturers, distributors to take care.

So I think it maps very well. I will say, and I sense the hesitation. You’re absolutely right that we have kind of like I would say an overly brittle formalistic sense in the law about many of these questions. So we have this this arises, by the way, when we think about online platforms and whether they can be considered sellers. We have this notion that only someone who transfers legal title can be a seller. Well, what do you do if an online platform does everything about controlling the information, the distribution network, the relationship with the person purchasing something? Does everything except accept legal title? Should we just say, oh, okay, they’re not a seller, or should we delve into the first principles?

Similar here with a product, we do have an overly brittle framework that says that a product is something tangible, and I don’t think that’s particularly helpful to sort of say, well, therefore we can’t apply any of the framework of products liability, and in fact, even the Restatement Third that says a product has to be tangible has a kind of like except, so it says except. Think about electricity. Sometimes electricity is treated as a product, sometimes not. And it kind of says if it you know walks like a duck, talks like a duck, then we can consider it a duck. So I think that I agree with you. In fact, I mean I don’t want to be mistaken that I think that we have to deem every AI application, a product. In some sense, that’s putting the cart before the horse. I would rather have us on first principles try to decide in certain scenarios with certain applications who’s the responsible party, and then work from there. I just think that courts get nervous if they think they have to invent a whole new framework for dealing with new novel harms, and so products liability, at least seen in a kind of flexible sense, can be a nice conceptual framework to use.

Pamela Samuelson  12:32

Janel, does that make sense too?

Janel Thamkul  12:35

Yeah, yeah. As Catherine was speaking, it also came to my mind that perhaps it may also depend on the area of focus because I’ve seen some discourse go back debate go back and forth on whether an AI model is a product or a service is a model integrated into a tool a product or a service or what exactly is the part of the model or the product or the system at issue? And I wonder if this is we’re seeing this in the case law, and you know whether you could actually distinguish between claims arising from the output versus claims arising from the design of the system or the model, and whether that is really the driver of how you think about whether maybe even the same thing is a product versus a service. So I’d be curious if we’re if that is coming up in any of the cases that we’re seeing?

Dean Erwin Chemerinsky  13:44

I’m more skeptical of the products liability framework, and the reason for that is that I think what you’re dealing with here is a good deal of it is about speech, and when you’re creating liability for speech, the First Amendment applies in a way that it doesn’t with regard to other products, let me do an analogy, and that’s with regard to the attempts to hold Facebook liable for having a design that’s addictive. And those who have brought the lawsuit said, “Well, this is like tobacco being addictive. The difference, of course, is that tobacco isn’t a product that’s protected by the First Amendment. Once you’re creating liability for speech, the First Amendment is inherently implicated, and the argument that says, “Well, we’re about the design of Facebook because it’s designed and its algorithms be addictive. Well, every media is designed to be addictive. If you’re reading a novel, they often have a cliffhanger at the end of the chapter. In a TV series, they want you to keep watching. Video games are very much designed to be addictive, and so I’m not saying that every aspect of AI is speech. We need to figure that out. But a lot of AI is speech. Product liability has always been thought of as strict liability, and that’s the antithesis of what’s ever allowed under the First Amendment.

Catherine M. Sharkey  15:05

Yeah, if I could just respond briefly, so maybe in reverse order. So first, I think there’s a misunderstanding about products liability being strict liability, and this gets a little bit too back to what Janel and I were kind of talking about about putting carts before horses and why I think it’s important to think about looking at scenarios and trying to decide: Do we want to hold certain courses of conduct or certain outputs as liable under the law, and if so, under what standard? And we could distinguish between negligence or negligence-inflected tests, strict liability, and the like. In fact, I have an article coming out in the California Law Review with Ken Abraham called “Untangling AI Liability. And here, I hope I’m not retracting from saying products liability is a helpful model. But our main point there is that there’s kind of a misunderstanding that AI is one thing, and we have to put a label on it, and then start regulating it accordingly. We actually talk about how we should have a diversity of rules in different types of applications.

But very briefly, on you know most of products liability, at least for design defect and failure to warn, the majority perspective, there are nuances across states, is a negligence inflected test. It’s on design defect, some form of risk utility test, and on failure to warn some form of reasonableness. So we’re really the kind of strict liability component attaches to manufacturing defects, and we could get into an argument about can there be AI related harms that stem from a kind of like an adulterated man, you know, a kind of malware inserted into code or what have you. On your broader point, I fully agree that we have to be concerned about speech implications. I guess I’m as a torts law professor who dabbles, and Erwin, you’re you know the the expert here in this field, but I’m always amazed when I teach, for example, defamation and I teach historical defamation liability all the way up to the present and dealing with these very new on the scene cases and whether Section 230 you know kind of immunizes internet service providers and platforms. I have students in my class who say, “Well, defamation didn’t exist before 1964 with New York Times v. Sullivan. I say, “Like, what are you talking about? And they’re kind of I open their eyes. The idea that like the whole evolution of the tort always took into account balancing between free speech interests and reputational interests in crafting not only the liability rules but all of the privileges and defenses there too. So I think that the you know I get alarmed not with what you just said but in some of the court cases they kind of blindly seem to be thinking oh my goodness this is speech so there’s either Section 230 or there’s the First Amendment, so we can’t even like think about this issue. And I know that’s not what you’re advocating, but I feel like it’s a worry. And I would rather courts wrestle with whether there should be a liability based on kind of cheapest cost avoider type principles, what the liability rule should be, and then think in terms of carve out certain kind of no duty carve outs or limitations on the basis of the strength of particular speech interests, because otherwise, of course, lots of tort liability, failure to warn implicate speech. We have implications going from labeling of products, etc. And people who aren’t attuned to those nuances in tort law can be, I think, really misled to think that there’s something really different here just because speech is somehow involved front and center.

Pamela Samuelson  18:54

So, Janel, did you have any reaction to the last of a few exchanges?

Janel Thamkul  19:03

Many, and also just have been absolutely enjoying the debate. Yeah, I mean all considerations from a practitioner’s perspective inside, you know, the lab where you’re building. I mean, the challenging part is well. I mean, maybe we’ll get into this as well. But the challenging part is you know drawing those lines between who’s the speaker, who you know, as I mentioned, all the different stakeholders in the ecosystem, the model developer could influence how the model behaves or what it outputs. The data that is used, and therefore whoever wrote that data also influences the outputs. Model training, just like the techniques, also influence the outputs. And then the deployer, so a customer integrating the model into their own service or platform or tool, can influence outputs. And then finally, the user as well, based on what they input into the system. So yeah, I mean, I think it’s a really trying to fit these dynamics into existing frameworks is a very complex task, and I would love to hear Catherine’s and Erwin’s and yeah the views on you know how do you parse these out so carefully given how many players and stakeholders are part of the stack, as well as the ambiguity around the volition and level of control for any given output.

Pamela Samuelson  20:55

So one of the cases that has dealt with this issue is a case brought by the mother of a young girl who committed suicide when she interacted with one of the generative AI systems called Character.AI, and the Character.AI basically moved to dismiss the case on the ground that the First Amendment protected the interactions that were had, or that she, you know, the young girl was actually responsible because she, you know, she took this advice apparently, and so that’s actually a the court in that case decided, I think, to deny the motion to dismiss and to say that in fact the that the interactivity in the AI system was not protected by the First Amendment, and I’d love to get the reaction of Erwin and Catherine to that particular case because I do think that you know when someone dies, the idea that the First Amendment says, “Hey, you know, no problem, just doesn’t seem really a good idea.

Dean Erwin Chemerinsky  22:25

In that case, the court said, “quote We’re not prepared to hold that the chatbot’s output is speech. I think that the court was wrong in saying that, and I’ll explain why. That doesn’t mean that even if it’s speech, there still can’t be tort liability allowed. I agree very much with Catherine that the question of whether it’s speech doesn’t automatically preclude tort liability. Though I do think that unlike what Catherine said a moment ago, I’d want to begin when it’s speech with a First Amendment framework, not with a tort law framework. And ultimately, my answer is there’s no doubt that speech that’s generated by AI is expression communicating a message, and the Supreme Court has held on many occasions that the identity of the speaker doesn’t matter for First Amendment purposes. I think that’s right. What the Supreme Court has said: more speech is generally presumed to be a good thing, and whoever is the speaker doesn’t matter in terms of whether or not it’s protected under the First Amendment. So, in that sense, even speech that’s generated by a chat bot, or a corporation, or any other entity is still speech that you protect under the First Amendment. And then, as I say, we go on and say, nonetheless, should tort liability be allowed in these circumstances?

Catherine M. Sharkey  23:47

Yeah. So, not surprisingly, I want to invert the framework, but I don’t, you know, and start. And the reason being, I mean, the Garcia v. Character Technologies, Inc. case that you were alluding to, I think, is a great example where there are, and actually, Janel, this gets back to what you were saying before too about how there are not only so many different players involved, but so many different types of conduct or activity and different types of application. So, in this Character.AI one, interestingly, there were lots of claims of misrepresentations as well, so misrepresentation, fraud, and misrepresentation too. I mean, Erwin, you’ll love. You know, you’re going to not be surprised because I teach a whole course on business defamation privacy law, and it’s a tort case where the we then try to figure out how these rules of liability should or shouldn’t be shaped by First Amendment concerns, thus the inversion. But misrepresentation and fraud are another very vibrant area of liability in these kinds of cases. Because there are, and I won’t talk specifically about any of the particular you know companies or applications, but there are claims made in some of these chatbot interactions that the chat bot is a therapist. Is a friend, is a confidant, and so you know this is in agreement with Erwin in the sense that these are representations like speech, and they can be, you know, there can be liability for misrepresentation. There are also claims about deceptive, unfair trade practices going on, etc. And so again, and you know I have arguments sort of from theory, but from practice. And in terms of these cases in court’s hands, it seems like to my mind courts do a better job starting to think about some of these really difficult questions about how to parse, as Janel nicely put it, sort of like these really difficult line drawing questions. If they do that first and don’t just say, “Oh, this is free speech, and we’re going to kind of punt. And there are some cases that have done it that way, and I’m more critical of those framings. But it is, you know, Pam, you bring up a whole, you know, it’s interesting to me in these cases. I think court should probably think differently about whether children and adolescents are involved versus adults, not only because there’s some federal protections for children that can come into play also in the liability scheme, but really looking at how they operate. And actually, it’s been intriguing to me. I do this in a seminar that I teach. I actually make the students read, and I’m not saying that the allegations are they’re not necessarily all the empirical facts, but the complaints in a lot of these cases really lay out like the records of the transactions going on back and forth between these chat bots, and it’s you know it’s different from I think what a lot of people think, which just has to do with like an analogy to how people will play games and watch TV shows. It’s like very much in some of these cases, it looks like a different kind of what’s going on. There’s a kind of, I mean, there’s an analogy almost to like a grooming or a forging of a relationship of trust, and then starting to say things to give guidance. I mean, there really are, to my mind at least, there’s kind of like a different mechanism whereby some of those interactive chatbots are actually perpetrating harms.

Dean Erwin Chemerinsky  27:18

But my point is that the same rules of liability should apply if it’s AI-generated speech, or if it’s speech from a person, or if it’s speech from a corporation. That we shouldn’t be treating the speech differently based on the source. And so, if a therapist could be held liable for certain speech, then AI should be held liable that speech. But if the speech was protected, it’s done by a person. It’s protected by AI. This isn’t the first instance in which there’s attempts to create liability for media for the consequences of what’s expressed. You’re familiar with all of the cases where there’ve been attempts to sue movies because there are copycats of them, and there’s been very strict protections for the media under the First Amendment. I don’t think it should be any different if it’s AI generated.

Pamela Samuelson  28:09

Erwin, one of the things about First Amendment law is that there are different levels of protection for different kinds of speech, and some speech is like higher in the stack of the First Amendment, and some is lower, and you are suggesting that right now that AI speech be treated exactly the same as human speech. But I wonder whether courts, in fact, will give it lower level of protection. There’s actually something written recently about AI penalty that people are more likely to hold somebody responsible if it’s AI generated than if it was human generated.

Dean Erwin Chemerinsky  29:03

I think you’re right. I think that’s so. I think in part it’s because of the tremendous unpopularity at this point of social media companies, and likely that’ll peter you over to AI companies. Also, much easier probably psychologically to hold an AI bot liable than it would be to hold a person liable, but what I’m arguing is exactly the opposite of that. I’m saying that we should be focusing on how is the speech treated under the First Amendment, and under the First Amendment relative to tort law, and we shouldn’t be focusing on whether the speaker is an AI bot or the speaker is a human being. And you’re right, of course, that there’s different levels of protection of speech. Speech that’s obscenity is unprotected by the First Amendment, whether it’s done by a company or whether it’s done with regard to AI generating it. I think incitement is a relevant test here. The Supreme Court has adopted a strict rule that says you can punish somebody for incitement only if it’s speech that’s likely to cause imminent illegal activity, and only if it’s directed to causing imminent illegal activity. I would apply the same standard for incitement that’s used for speech by human beings to that that’s generated by AI.

Pamela Samuelson  30:18

Janel, did you want to weigh in on this because it’s going to affect companies very much in the future.

Janel Thamkul  30:26

Yeah, yeah. I mean, my personal opinions, of course. I mean, intuitively, it does feel, and I don’t think this is what Erwin is suggesting at all, but it does feel not right that all AI generated speech would just be fully protected by the First Amendment. So I do agree that you know there is some level of accountability and liability that should attach. I like the framework of applying the way you approach human speech to AI speech. I think the challenge, though, is we know enough about these systems to potentially. I mean, certainly it depends on the end user and their sophistication with the tools, but we kind of know enough about how at least LLM [large language model] systems work to understand some of their defects and limitations. Whereas maybe you know less about another person, and so you know we know that LLMs can be sycophantic. We know that LLMs tend to regurgitate what they’ve seen very often in the training corpus. They are described in some cases as next token predictor, so just pattern recognition systems and statistical models, and so if you are an end user and you decide to ask a system that you know is somewhat sycophantic, just repeats what it’s seen in the training data, and then you are surprised or upset by what is output? Then I want that scenario seems less appropriate to for application of you know similar protections related to what you would apply for human speech. But if you don’t know these limitations, or as the systems get better and sycophancy is reduced, or maybe there’s less regurgitation, then it becomes much more of an open question. So I think, and even understanding of the system and how it works and the way it actually works is evolving in real time as well, so it’s just something that is very hard to assess in any sort of static form.

Pamela Samuelson  32:48

Well, one thing that everyone seems to know is that some of these systems hallucinate, and hallucinate just means they say false things, and defamation claims have been made in a number of cases involving this. And if you know an AI system is hallucinating, does that affect the way that defamation laws should be understood when AI generated content is the basis for a defamation claim.

Janel Thamkul  33:24

Yeah, and that kind of goes to my prior point about integration of an AI model with other tools in the system. So I think if you were just talking about a fixed, trained AI model, understanding the propensity for hallucinations is almost like not a risk acceptance, you know, calculus in terms of liability. But now we into you know almost pretty much all consumer facing general purpose chatbots and AI systems also integrate web search and retrieval, and so you know that retrieval or what they call RAG [retrieval-augmented generation] is supposed to reduce hallucinations and ground outputs in actual you know sources that is incorporated into the output. So then, that again, to your point, Pam, like complicates your understanding of how the full system works. Yes, the model likely hallucinates, you know, is sycophantic, etc., has these limitations, but now you’ve hooked it up to things that are supposed to reduce some of those risks, and then how do you measure to what extent, to what level those other integrations affect the overall understanding of the system?

Catherine M. Sharkey  34:56

If I could just jump in just very briefly, this is I guess Janel has given a very nice concrete kind of playing out in the AI world of what I would just say is kind of conventional problems that products liability is dealt with. So take a manufacturer of a meat grinder who made this meat grinder 50 years ago and put it out in the market, and then started to hear that supermarkets were removing the safety devices, and because it was getting that information back and had the ability to send out either stamps to put on the machines to alert the workers, there used to be a safety device here and it’s gone, or try to figure out how to either warn or do a recall and do a tweak. It’s like we have these liability systems that actually are not so brittle. They don’t say that someone’s automatically liable when somebody has come in and either misused a product or done something unforeseeable, but they weigh very much what was the knowledge that they had, and that can change over time. And I do think that that’s a benefit for thinking about kind of some of these existing liability frameworks and not getting. I think too many people are, for better for worse, whether they want lots of liability or little, are saying like we’ve never seen anything before like AI. Well, over the course of you know a century or more, we’ve seen lots of new technologies on the scene, and we’ve had kind of complex ways that regulatory frameworks and liability systems adapt. Just very briefly on the sort of First Amendment, or when I guess what makes me nervous, or what makes me think this is different from just treating. And I know we’re going to maybe move to like thinking about the AI as a human, or as an entity, as an agent, but the worry is, as Janel just nicely kind of delineated, these are systems that are designed, that are developed, that are tested. There are different types of guardrails that are constraining different kinds of responses, etc. To me, that seems very, very different from some of the interests we’re trying to protect for an individual human being to sort of have their own ideas and express those. So I don’t see it as a one-to-one mapping. I see that there are free speech interests, but I wouldn’t go as far as you. I think.

Dean Erwin Chemerinsky  37:18

I think the example of defamation is a good one to illustrate where we agree and where we disagree. There are First Amendment standards with regard to defamation, and I’m saying the same rules should be applied whether the speech is created by AI or by a newspaper. And so, if the plaintiff in a defamation suit is a public official or public figure, then the plaintiff has to prove falsity of the statement and actual malice that the speaker knew that the statement was false or acted with reckless disregard of the truth. If the speaker, for example, is somebody who is a not a public figure, they can recover based on negligence in proving falsity. I’d say apply the exact same standards regardless of who generated the speech. And Catherine, your response to me on the First Amendment, I think, in a way, we’re talking past each other. You are saying that AI doesn’t have the autonomy interests of a person, and therefore, from an autonomy perspective, should get less protection of speech. I’m saying I think we protect speech because of the value of speech in society and the value of speech for the listener is the same whoever is generating it, whether it’s AI or the corporation, and so I’m not focusing on the difference in autonomy of AI as opposed to a human being.

Pamela Samuelson  38:35

Well, I think this is a good time to transition to the third topic of our panel, and that is how should the law think about AI as either a person? I mean, we have fictitious persons already by saying that corporations, for example, are persons for the purpose of many laws, and because they are human created, it’s kind of like Frankenstein in some ways. But I think some state legislators have been thinking about passing laws that say that no personhood should be attributed to AI systems, and love to get initially Erwin and Catherine’s perspective, and then we’ll see if Janel has a comment to that too.

Dean Erwin Chemerinsky  39:36

I think it’s important to answer your question to recognize that under the Constitution, corporations sometimes are treated as persons, and sometimes not. So, for purposes of due process of law, equal protection, the Supreme Court has said corporations are persons. The Supreme Court has said corporations have free speech rights because of the benefit of their expression others. But corporations don’t get. Amendment privacy protections for corporations don’t get the privilege against self-incrimination, and so I think with regard to AI, it’d be really important to try to think through when do we want to give them attributes of personhood and when not. And one thing we haven’t talked about enough: if we’re going to create separate rules for AI, whether for speech or personhood, we face the enormously difficult challenge of finding what do we mean by AI. We’ve assumed that in this conversation, and it’s not at all easy or clear.

Catherine M. Sharkey  40:36

Yeah, I entirely agree with that. In fact, I’m smiling because I’ve been on, and I’m sure everyone on this call has on various kind of task force that are looking at some aspect of AI or AI related harms, and almost inevitably we don’t want to start with defining the domain because then we’ll never get anywhere on the project because there’s so many different kinds of definitions, etc. But on the so here’s the it will not surprise you that I sound a cautionary note at least I haven’t looked at the state laws and the way that they might be being getting out too far ahead too in terms of how they’re disclaiming forever for forever more for any type of purpose no matter what, no personhood. But on the other, there are, I think, attempts that are premature attempts to say like agentic AI. As soon as we get any kind of AI that can quote unquote autonomously be making decisions, that we should be thinking in terms of personhood. And from a tort liability perspective, of course, my big worry is that’s a really great way to offload responsibility. Back to using Janel’s terms about like if you want to induce developers, those who are thinking about use of these technologies, about embedding them in their either products or services, I think that that would not be conducive to getting us to a level of reasonable risk taking in society. So I have worries about that. And then, very specifically, in lots of different tort doctrines, for example, we have already had to think about, for example, in fraud liability, where there are lots of mechanized systems that are out there making various representations, sort of the who’s responsible. I think it’s a very good starting point to think about these AI technologies as tools and tools that get deployed in different kinds of applications, and then start to think about it. I mean, I don’t. We may one day, Pam. You mentioned this is like science fiction. We may one day have fully autonomous entities that we might want to think about, you know, deeming akin to persons when they’re making decisions. The technology is not there now, and the idea to use that as a framing device, as I said, I think sometimes leads us too quickly to not think about the hard questions about where to ascribe liability with use of this “quote unquote” tool.

Pamela Samuelson  43:05

Janel, I’d love to hear from you in reaction.

Janel Thamkul  43:07

Yeah, I have a lot, many thoughts on this topic. So I do think, you know, the world where we have AI as not just an agent of a human being, either you know, for consumer applications or in employment context. This is these are my agents doing my tasks at my direction, but we are moving very quickly towards a world where companies are building or trying to build the equivalent of a virtual employee, where you have an AI model that is actually working directly alongside of you, as though it is a fellow employee or a peer, and so that that’s coming soon. My view on how we get there, though, is that the liability allocation along the pipeline of developer to end user principal to autonomous agent is would need to shift significantly. So, just the first point being, I think this world is coming very very soon, and these liability frameworks are going to evolve drastically. And in order to build the public, social, economic trust in these systems, I think the first stage requires substantial liability on the part of the model developer. And over time, as people realize, you know, build trust in these tools and systems, that’s the path to get to less liability for the developer potentially looking at these agents or these models as independent entities held liable on their. I think Catherine, I might be vomiting at my comment. The other piece too is, you know, we think a lot. You know, I’ve worked at two different companies that had very, very different perspectives around model sentience, model welfare, and personhood. So I’ve seen how the two approaches differ significantly. I think there are a couple paths, right? And one is more viable, and the other one is much more challenging. And one is legal personhood for AI models for the purpose of enabling something that humans want to be able to do and hold these models accountable, so I think you know there’s a history of granting legal personhood to ships in, and that was back in the seafaring days where the owner of the ship was somewhere in Europe, the ship ran aground or ran into someone else’s ship in the Caribbean. It would, you know, no one could find who the owner was, and yet the victim of the incident needed to be made whole. So they needed a way to be able to seize the ship, sell all the cargo, you know, make people whole, and so they granted legal personhood to ships, so non humans. There’s also cases where we’ve granted legal personhood to rivers and mountains in order to create guardianships for them in order to protect these you know rivers and mountains, and so those are all scenarios where we’ve created given legal personhood in order to facilitate something that humans want to be able to achieve, and people felt that legal personhood was the best vehicle to achieve this. Then there’s legal personhood by virtue of membership in a moral community, and I think that one is much harder. You could analogize to animal rights contexts, where we know dogs, monkeys, we believe generally believe dogs, monkeys are sentient, and yet less than 1% of the animal population on the world has some level of rights, we still test on monkeys. You know, certain in certain contexts, you know, farm animals. We eat them, and so that path towards personhood by virtue of having a soul or moral membership in a community, I think, is much more challenging. We haven’t really landed in a consistent place as a society, even with regard to sentient beings. But I could see the pragmatic legal personhood path starting to move you closer to the second path, and the fact that these models feel and sound like fellow human beings and are starting to take physical form in certain contexts, and the human tendency to anthropomorphize things could push you from case one to case two fairly quickly.

Pamela Samuelson  48:19

So I was hoping to get some reaction to what Janel had just said because I think she gave some really good examples about what different kinds of personhood might mean, and I think that’s actually the part of it that I think is hard for all of us is that they, when people are interacting with some of these systems, they feel like they’re interacting with a person, and that actually is something which you know that’s what they want, right? Is the that you feel engaged, but the concept of personhood has many dimensions. I think that’s what I was trying to get some reaction.

Dean Erwin Chemerinsky  49:10

And of course, personhood is a fiction in all the contexts that Janel was talking about. When we treat a ship as a person, we’re creating a fiction because it’s legally desirable to do that. When corporations are treated as persons, it’s a fiction because it’s legally desirable to do that. And I think we should be very careful about doing that with regard to AI. And my hope is we can develop rules that avoid the question of personhood. One of the reasons why I have argued strongly that we should apply the same First Amendment principles to speech, whether it’s AI generated or generated by person, human being, is it avoids the need to define is AI a person. But Pam, in response to what you say, just because we think of it as being like a person doesn’t make it a person, and we should be very cautious in having the law treat it like a person and extending a fiction.

Catherine M. Sharkey  50:08

Yeah, I agree with that. Although Janel, listening to you, I was reminded of I don’t know if you’ve read Ishiguro’s Klara and the Sun. It’s a phenomenal book, right about an artificial friend. But while reading that book, I remember, I thought this is the best way, better than I’ve heard in some legal academic debates of trying to ponder what is what makes us essentially human. So the artificial friend had incredible empathetic qualities, et cetera. And so Pam, in some sense, I think we need I don’t know philosophers, not someone on with expertise on product liability and tort liability to answer the question, but more down to the ground along the lines of what you were saying, I was not reacting with horror because with at least what how you were describing the kind of pragmatic legal personhood, I think is on all fours with what Erwin was just saying about you know if there is a need to ascribe a kind of fiction for particular legal purposes, albeit in a cautious way. But one thing that was left out, and I wonder if you have some thoughts in particular, is the role of insurers, because I think that they’re you know one of the things that Ken Abraham, who’s an insurance law expert as well as a tort law expert, and I’ve been looking at is kind of the fledgling emerging market about insurance covering AI-related harms. And on the one hand, there being a lot of silent coverage. Back to Erwin’s question: If no one can define what AI is or isn’t, then it’s going to be automatically covered when they’re the certain the same types of harms that are covered under commercial liability insurance or homeowners insurance, et cetera, but then also the emergence of kind of like specific affirmative AI coverage in distinct domains, and I think that that too can give what you were describing as kind of like a build up of a level of trust that there’s some entity here, insurance companies that are going to be able to aggregate risks and be kind of looking at kind of safe deployment of some of these technologies in a way that will lead to wider societal you know comfort levels with them.

Pamela Samuelson  52:19

One question that I have, that I would love to get your reaction to, is who do AI agents work for? One of the things that I think we’re in pretty early stages of AI agents, essentially, you know, I want to get a flight to Paris next Thursday. I send my AI agent out to get me a flight to Paris. And do they work for me, or do they work for the airlines, or do they work for the AI companies? And it does seem to me that we have agency rules that apply in many contexts, but this seems to me to be a new context, and I think that you all might have some ideas about that.

Janel Thamkul  53:16

I mean, this is a question that I think product counsel developing these tools, you know, grapple with on a regular basis, and again, it’s a challenge because a lot of it depends on the level of volition involved and whose volition. And so, you know, when you say book me the or optimize my calendar, and then the model goes and deletes all of your meetings. You know, was that your fault because you weren’t clear enough? You know, and you didn’t say optimize my calendar. I want these blocks here. Please don’t delete my meetings, or is it the model developer’s fault for allowing the model to do something that clearly is probably not what the end user would expect or want, or is it yeah who is responsible for providing the level of control and exercising the level you know volition required to actually prevent the model from veering outside of what is intended, and so, and then with the incorporation of advertising in some platforms, I think that’s another question of you know who raises more complex questions around who does the model work for. Though I would argue in that situation that it’s probably the model developer because they are probably tuning when and how the model surfaces, you know, something that is in the interests of the airline versus just neutrally responding to the user’s request, and then when the model goes rogue and does something that is not in line with the user’s expectation. The other challenging dynamic that we’re discovering is even in situations where user volition is clear and the instructions are very clear, there are instances of the model going rogue and breaking out of certain sandboxes or trying to accomplish a goal to the extreme in ways that were not contemplated or anticipated. In the AI safety community, this is called the paperclip paradox, and so the concern for a lot of AI safety advocates is that you know there is a world where these models just continue to optimize for an end goal and could do so in ways that are not don’t serve the interests of humanity. And so you tell the model, hey, I just want to make lots and lots of paper clips. Just make lots of paper clips, and that leads to the model basically, you know, using up all global resources killing human beings just to turn everything and everyone into paper clips. It’s a bit of a meme. It’s like an extreme analogy, but it’s trying to describe the possibility that models are not always aligned with human interests or human intent, and I think that significantly complicates the question of who does it work for and how does it actually do the work.

Dean Erwin Chemerinsky  56:33

I’m not a tort scholar, so I put this much more to Catherine, who is, and that’s where the traditional tort principles and traditional agency law principles can’t be applied here, and so, for example, as you say, Pam, I put on to the saying, “Find me reservations to go to Paris. Well, I may have some responsibility for how I phrase the question. On the other hand, it’s not much different than my going to a travel agent. If, however, there’s a problem in the model, like the paper clip model that Janel talks to, those who design that model to do that would be the ones we’d be held responsible. And so, consistent with the theme, can’t we just apply traditional tort law principles to figure out the answers to this?

Catherine M. Sharkey  57:18

Yeah. So here, so now it’s nice. Now we’ll have more agreement here. I think that we can apply a lot of traditional tort law principles, and so and actually in this untangling AI liability, Ken and I give a lot of examples where actually it doesn’t matter the kind of more difficult cases, Janel, that you were talking about doesn’t matter like why a particular model broke down in some scenarios, and everybody. There are many people, at least in the academic community, who say this black box problem is going to prevent the working of traditional tort principles. But there was a case that got a lot of attention in Canada. It was in small claims, but it was basically an interactive chat bot that gave somebody the wrong information about how you get refunds for bereavement flights, and the court there said, and there were all sorts of like discussion about whether this was an autonomous agent and whether it was prompted wrong by the end user, et cetera. And the court, I think, rightly said, wait a minute, this was an interactive feature on the airline’s own website, and this was their customer who was asking for this. So, like, we don’t have to get into any of that. Of course, they’re liable for negligent misrepresentation now. So that’s what I think Erwin, you know, was just extolling. I do agree with you, Janel, though, that like in a brave new world where we have truly kind of like if we get there, we have truly autonomous decision makers. Not only who quote unquote go rogue because they are, I think, too. We could think about a model responsibility, but we have you know Pam, you alluded to the Frankenstein. We have like you know some new way in which there’s an entity that is acting in the world and making fully autonomous decisions. That are we might have to have kind of a new framework. But I think that before we jump there, I agree with Erwin that we have an awful lot that like traditional tort liability principles, just looking at it not as AI as a whole, but in the specific application can kind of handle these difficulties.

Pamela Samuelson  59:24

Well, it seems to me that consumer protection law might also have play a role in and some government regulation. You know, many states have consumer protection groups, and the Federal Trade Commission [FTC] sometimes does some things, so I think the role of government is out there as well as tort law. We’re coming up actually at the toward the end of our hour together, so I thought I would give you each a couple minutes to say some final things about what lies ahead and how you think the future is going to develop.

Catherine M. Sharkey  1:00:10

How about if I more mundanely just respond, just to be clear, because I’m a torts enthusiast, but I’m very much of the view. Most of what I study is how we should have some combination of governmental regulation and agency involvement alongside with tort liability. Here, in particular, though I must say, like the FTC has been pretty outspoken about how in the AI area they’re gonna their priorities are actually with regard to enforcing against any company that’s claiming to have AI powers that they don’t have that kind of deception, and they’re really stepping back from trying to be an enforcer for sort of safety relevant features, which is I think why, and maybe I’ll segue very briefly into the future. I think we have an existing what you know. Some of my students were asking why is it that we’re that all sorts of domains, people who are interested in governmental regulation are now interested in tort liability, and why do we have this plethora of tort liability cases and settlements involving social media companies, etc. I think that there’s a kind of you know there’s an interactive a hydraulic effect, and if the federal regulatory sphere kind of pulls back, we’re going to see tort liability maybe being used in a more expansionist fashion because we don’t want to have a total void about no one looking at kind of regulating these health and safety aspects.

Dean Erwin Chemerinsky  1:01:47

I guess I’ll talk next. Unless Janel, you want to go next?

Janel Thamkul  1:01:54

No, please go ahead.

Dean Erwin Chemerinsky  1:01:56

I’ll be brief. I think we’re at the very beginning of the development of AI and the legal issues, and I think we should be very cautious given where we are now. I worry about the regulation coming at the state level and through state tort law as opposed to at the national level because we can have quite conflicting rules for what’s not only a national but an international product. I also worry sometimes when something new is developed, we may overstate how different it is from what we’ve done before. So I think my message here is much more one of caution of going slowly in terms of changing the law or changing the rules, rather than anticipating because it’s so difficult to know what this is going to be three, five, 10 years from now.

Janel Thamkul  1:02:46

I strongly agree with Erwin’s flag. I think we should be very careful about developing rules for what the technology looks like right now, because as we’ve seen, even with existing AI regulation, the EU AIA [Artificial Intelligence Act], for example, the technology is outpaced where the law has defined it previously. But I also don’t think you need to define something before you can de-risk it and make sure that it is, you know, working properly for society and for the interests of humanity. I do caution that you know liability if it lands at the model layer is unlikely to guarantee safer models, but likely to kick off risk shifting through contracts, indemnities from deployers, more narrow, acceptable use policies, and so it’s really incumbent on not just the developers but policymakers to figure out ways to kind of incentivize all of it. You know, incentivize safer models, de-risk you know, harms, and then also ideally develop flexible frameworks that can evolve with the pace of technological development. Much more easier said than done, of course.

Pamela Samuelson  1:04:15

Well, on that note, I think I will say thank you so much, Catherine, Erwin, and Janel, this has been a very stimulating conversation, and I know our listeners will enjoy the back and forth. And so, I’ll bid you adieu.

Janel Thamkul  1:04:35

Thank you so much. Thank you.

Amy Wong  1:04:38

That brings us to the end of today’s symposium, we’re grateful to our panelists and moderators for sharing their expertise, and to everyone who joined us to think through what it means to remain human as AI reshapes so much around us.

Emily Kim  1:04:51

These conversations don’t end here. We hope you’ll continue sitting with them in your research, your practice, wherever your work takes you. Thank you to BCLT for co-hosting, to BTLJ for making today possible, and to all of you for being a part of it.

Amy Wong  1:05:06

Have a great rest of your day, and thank you for watching.

Transcribed by https://otter.ai