Day 2, Panel 5 (Litigation): AI Tools and Litigation
December 5, 2025
B-CLE Recording (CLE: $100) | Agenda | Event Resources | Speaker Biographies
Download Panel Transcript & Materials Here
Speakers
Sharif Jacob, Keker, Van Nest & Peters
Eric Lin, Patlytics
David Lisson, Davis Polk
Sasha Rao, Nixon Peabody
Panel Resources
APLI 2025 Litigation Panel – AI Tools and Litigation slides
Panel Summary
Chris Mammen opened by surveying the room on AI adoption, finding near-universal use among the patent litigation audience — a notably higher penetration than the 30% reported by the ABA’s 2024 survey of legal practitioners broadly, which itself had tripled from 2023. Eric Lin provided market context, noting that ChatGPT has 800 million weekly active users, that Anthropic has reported 800% growth in annualized recurring revenue, and that Harvey — an AI legal tool still technically a startup — recently raised over a billion dollars and now counts 42% of AmLaw 100 firms as customers. The panel then moved through concrete AI use cases, with Sasha Rao highlighting invalidity claim charting and opposing brief analysis as two of her highest-value applications, while Sharif Jacob noted that AI claim charting tools had surprised him by generating novel theories of invalidity beyond simple element-by-element mapping, and that AI was also proving useful for oral argument preparation by generating likely judicial questions. David Lisson emphasized that the ideal AI use cases share a common profile — large datasets, routine tasks, and limited need for creative legal judgment — and flagged an emerging and cautionary trend of clients using AI as “shadow counsel,” bypassing outside and in-house counsel entirely to get answers directly from ChatGPT, often creating downstream problems that increase legal fees rather than reducing them. Chris Mammen added that AI is particularly powerful for expert deposition preparation, describing how he used it to rapidly identify which of a 200-page expert CV’s publications were relevant to patents in suit — a task no client budget would support through traditional associate review.
The panel then addressed court-specific AI rules, ethical obligations, and the future of the legal profession. Sharif Jacob walked through the patchwork of judicial standing orders across key patent litigation jurisdictions — including Northern District of California judges requiring lead counsel to personally verify all AI-assisted citations, Eastern District of Texas requirements tied to Rule 11 certification, and the Western District of North Carolina’s blanket requirement that every filing include a certification that no AI was used except through approved legal research platforms. David Lisson and Chris Mammen flagged the growing tension between outside counsel guidelines — many of which contain AI prohibitions drafted in 2022 or early 2023 before enterprise-grade protections existed — and the current expectations of in-house legal departments who want their outside counsel to use AI to reduce costs. Eric Lin urged practitioners to carefully evaluate AI vendors beyond surface-level certifications like SOC 2, noting that security, workflow integration, and the specific steps of the litigation lifecycle a tool supports are more important differentiators than marketing claims. On the profession’s future, the panel was divided — Sasha Rao warned of significant labor market disruption particularly for junior lawyers, while Sharif Jacob offered a more optimistic view that AI would eliminate onerous grunt work and allow lawyers to handle more matters more efficiently, and David Lisson raised the deeper pedagogical challenge of training the next generation of lawyers to develop sound legal judgment without the foundational repetitive work that has historically built it.
Key Learning Points:
-
AI’s Litigation Sweet Spot Is Large-Scale, Routine Data Tasks: The highest-value and most reliable AI applications in litigation involve processing large volumes of structured data with limited need for creative legal judgment — including invalidity claim charting, portfolio mining, pre-litigation diligence across hundreds of patents, opposing brief gap analysis, and expert CV relevance screening — while AI remains unreliable for tasks requiring nuanced legal strategy, such as selecting which patents to assert or drafting full briefs (David Lisson; Sasha Rao; Eric Lin; Sharif Jacob).
-
Court-Specific AI Rules Are Now a Threshold Compliance Issue: A growing number of federal courts — including multiple judges in the Northern District of California, Eastern and Northern Districts of Texas, and the entire Western District of North Carolina — have issued standing orders governing AI use in filings, ranging from general Rule 11 reminders to mandatory personal lead counsel citation verification and blanket AI certification requirements, making jurisdiction-specific rule review as essential as checking local rules when entering any new matter (Sharif Jacob; Chris Mammen).
-
Outside Counsel Guidelines and Engagement Agreements Are the Hidden AI Compliance Minefield: Many outside counsel guidelines contain AI prohibitions drafted in 2022–2023 that conflict with current in-house client expectations and internal firm AI adoption, while engagement agreement fine print raises unresolved questions about data ownership, precedent rights, and vendor attestation obligations under protective orders — creating a complex compliance environment that requires coordination among lawyers, IT, data privacy teams, and clients before any AI tool is deployed on a matter (David Lisson; Sharif Jacob; Chris Mammen).
Program Transcript
Key Terms: AI Adoption Rate, ABA Survey 2024, McKinsey AI Survey, Generative AI Penetration, Weekly Active Users, Annualized Recurring Revenue (ARR), AmLaw 100 Adoption, Harvey AI, OpenAI, Anthropic, ChatGPT, Claude, Gemini, Enterprise AI Tools, Legal AI Market Growth, Point Solutions, Walled Garden Instance, Invalidity Claim Charting, Prior Art Claim Charting, Portfolio Mining, Pre-Litigation Diligence, Opposing Brief Analysis, Brief Gap Analysis, Oral Argument Preparation, Expert Deposition Preparation, Evidence of Use Investigation, Infringement Analysis, Patent Assertion Selection, Validity Charting, License Letter Response, Invalidity Contentions, Competitive Landscaping, AI Tool Selection, Security Evaluation, SOC 2 Certification, ISO Certification, Workflow Integration, Sandboxed AI Instance, AI Governance Committee, Mandatory AI Training, Prohibited Uses Policy, Acceptable Uses Policy, Vendor Attestation, AI Evangelists, IT Collaboration, Data Privacy Review, AI Policy Framework, Standing Orders, Local Rules, Rule 11 Certification, Lead Counsel Verification, AI Certification Requirement, Hallucinated Citations, Hallucination Tracker, Western District of North Carolina, Northern District of California, Eastern District of Texas, Northern District of Texas, Central District of California, Judge Martinez Olguin, Judge Kang, Judge Lin, Judge Lee, Judge Kong, Judge Kacsmaryk, Judge Baylson, Duty of Competence, Duty of Diligence, Candor to Tribunal, Attorney-Client Privilege, Attorney Work Product, Supervisory Responsibility, Independent Judgment, AI Disclosure Obligation, Citation Verification, Human Review Requirement, ABA Model Rules, California Rules of Professional Conduct, Outside Counsel Guidelines, Client Engagement Agreements, Shadow Counsel, AI Prohibition Clauses, Data Ownership Questions, Client Consent, Little C Client, Big C Client, In-House Legal Department, AI Cost Reduction Pressure, Client Bill Review, Engagement Agreement Fine Print, Protective Order Compliance, AI Protective Order Provisions, Vendor Attestation Requirement, Sandboxed Confidential AI, E-Discovery Order, Discovery Database Integration, Confidential Information Protection, Opposing Counsel Negotiation, Labor Market Disruption, Junior Lawyer Training, Legal Education Impact, Industrial Revolution Analogy, Do More With Less, Fixed Fee Work, Grunt Work Automation, In-House Role Stabilization, Associate Development, Karate Kid Learning Method, Sedona Conference AI Working Group, Legal Education Reform, Next Generation Lawyers, Womble Bond, Keker Van Nest & Peters, Patlytics, Harvey AI, McKinsey, Thomson Reuters, ABA, California State Bar, Sedona Conference, Coinbase, Waymo, Paul Grewal, Amar Mehta, Law360, Westlaw, Lexis, FastCase, Bloomberg
[HOST]
Okay. Thanks for making it back after lunch, everyone. We’re moving on to AI tools in litigation and how we can save the clients in the room lots of money, maybe or potentially make things more complicated according to at least one panelist yesterday.
Um, with that, I’m gonna turn it over to the moderator, Chris Mammen from Womble Bond.
[CHRIS MAMMEN]
Great. Try this. One way or another, we’ll get the mics shared.
Um, so I’m really pleased to be moderating this panel on the, the newfangled tech CLE credit, and I’m joined today by a– a fantastic panel. Uh, we’ve got Sherrief Jacob, Sacha Rao, David Listen, and Eric Lin, and we’re gonna try and uh, make your blood run cold. So why don’t we start off thinking about the state of AI in legal practice?
And we’re gonna do a show of hands here, and I want, I’m gonna go through several things, so just keep your hands up until, as long as it applies. So show of hands, how many have used generative AI in some fashion? Everybody in the room. For work?
Once a month? Once a week? Daily? So, all right.
It sort of drops off as we see that. It’s a pretty high penetration level. What we’ve seen in terms of statistics are that is a much higher representation than we see across the practice generally.
And Eric, do you want to kick things off with some, some data about adoption trends?
[ERIC LIN]
Yeah. Sure thing. So I think the unique aspect of the audience here is that we are patent lawyers, patent litigators, and so we’re naturally considered to be tech-savvy.
So since the statistics here look to be like near 100%, but in reality in the legal practice the ABA conducted a survey in 2023 and 2024, and for the 2024 survey the percentage was 30% of attorneys using AI tools in practice, and that tripled between 2023 and 2024. They haven’t done 2025. I assume it’s probably going to be at least double, above 50%. But I think what’s like important about all of this is that in the legal industry, it’s traditionally been a little bit more conservative and slow-moving.
And so a good thing to kind of compare how adoption in legal with AI is, is to see how businesses generally have adopted AI, and so McKinsey does a pretty regular survey of business organizations, and in 2025 they found that almost 80% of respondents from major business organizations reported the use of generative AI in at least one function. So we’re comparing like 80% versus like the 30% that was shown in 2023 for legal practitioners, and so there’s still a little bit of a gap there, obviously. I think we’re gonna see a lot more of it, especially by the end of this year given the amount of tools and software that’s available.
But you have to remember that for ChatGPT, they are reporting that their weekly active users is 800 million, so 800 million weekly active users of ChatGPT. And we’re talking about OpenAI and, and Anthropic with their LLM models going through historic growth in terms of annualized recurring revenue. Uh, we’re talking about like, you know, venture capital-backed growths, and you then have to compare that to like, say, a traditional kinda company like Salesforce.
And I think Anthropic specifically reported 800% increase in ARR, and you compare that to like Salesforce, which is very also a technology company, but they report an increase in 8.7%. So this is all just to show adoption is increasing in legal, but it lags a little bit compared to the broader sector.
[CORY MAHONEY]
Great. And so al– also, in the in the ABA report that, where, where Eric drew the numbers from there’s some indication of the, the areas where AI is being used and those are up on the screen.
Um, there’s also uh, so– a more recent survey from Thomson Reuters showing among those who use generative AI in litigation these are the, the top eight use cases and, and the prevalence of those. And we’re gonna talk a little bit more in detail about some of these use cases as we go forward.
The expectation is that with as many of you as raised your hands are using this for work, you’ve got your favorite use cases, you’ve got things you do. But hopefully through our conversation today, you’ll come away with some other ideas of things you want to try or inspiration to go figure other use cases out.
The key value adds include you know, in terms of picking use cases handling large volumes of data, speeding things up making it more accurate, and providing advanced analytics. And sort of leading then to the next thing that I thought we should talk about is who is using, among the legal industry, what do we think the trends are in terms of what we’re observing?
Big Law? Solos? Tell me what you’re thinking. Um, David, do you wanna–
[DAVID LISSON]
Yeah, I can give the Big Law perspective, which I think won’t surprise anyone that Big Law is more conservative, but also has the resources to put some money into either their own internal AI sort of tools or using external tools and trying them out. I would say one nice part about, you know, being in a big law firm is you can set up, so long as you have a committee that is making sure everything is compliant with all the ethical obligations and client obligations, which we can talk about later, the size gives you the opportunity to experiment with different things and spread that learning across the firm.
And so, you know, you definitely need some oversight, a lot of oversight because of the risks we’ll talk about later. But, you can set up little trial projects and get the benefit of those trials and then decide as a firm what you’re gonna do, sort of, and release more broadly.
So that’s one way, in addition to the obviously financial backing, that you can throw at these new tools. That’s a nice part about Big Law, I think. Probably balancing that against the risks and the sort of more cautious nature of law firms generally.
[CHRIS MAMMEN]
Any of you wanna sort of weigh in on perspectives of other than sort of the Big Law firm adoption.
[SHARIF JACOB]
Yeah, sure. I’ll provide two perspectives.
I think at our firm, I’m a partner at Keker, Van Nest and Peters, and we’re more of a boutique size. And we have a committee whose job it was to go out and look at all the AI tools and figure out what would be useful to use and what wouldn’t be useful. And then they tested a bunch of them with IT and figured out would they meet confidentiality requirements, would they be sandboxed, et cetera.
And then we developed a policy that would apply firm-wide and there’s sort of a mandatory training. Everybody who’s gonna use AI has to go to this mandatory training and then sort of sign under the policy. And it’s got prohibited uses and acceptable uses.
Our firm doesn’t allow you to write a brief using AI and soon enough we’ll talk about why that is. But it encourages you to use AI for all sorts of stuff. Our firm requires that if you’re gonna use AI, you have to be candid about it. So if you’re the associate, you need to tell the partner that that’s what you’re doing.
And then the policy also focuses on, you know, communicating with the client and compliance with protective orders. So that’s sort of what some of the more, you know, medium-sized firms are doing. One other statistic I’ll throw out, I think about three months ago I saw in Law 360 that 25% of law firms, not just IP firms, 25% of law firms had purchased claim charting software. So I do think that you know, AI is coming to the firms.
Yeah.
[SASHA RAO]
So at our firm we have some of the big commercial tools for AI that other firms have, but we’ve also built our own internal tool for AI which safeguards confidentiality and it allows you to use any you know, any model, ChatGPT, Claude Gemini, any model within that sandbox. So you can run your queries and say, “Use this model to get me the answer.” And it’s internal to the firm, but it’s very generic. So I like that.
[SHARIF JACOB]
Cool.
[ERIC LIN]
Let me just add a little color with some statistics, additional statistics here. Um, Harvey, which is still called a startup they recently raised more than a billion just alone this year and are at a $8 billion valuation. They’ve reported that 42% of Am Law 100 firms have adopted their software.
So that’s a interesting metric given how some big law firms can be seen as conservative. But obviously there’s client pressures and then the need to be satisfying like leaders in terms of technology. And the second thing is, to echo what Sharif says uh, my company we offer patent AI solutions and we’re also seeing near 40% adoption by AM Law 100 and also top like IP litigation firms.
So very similarly that kind of echoes where we are seeing in terms of like the trends up.
[CHRIS MAMMEN]
So, you know, one other observation is because you know, some of the free AI platforms are so good at some of this you know, small solo firms may be able to or may be taking advantage of some of those capabilities without the kind of bespoke legal industry investment that we’ve heard about from our panelists. And you know, we’ll have a chance to talk into some of the advantages and disadvantages of that. What I’d like to do now is, is to talk about use cases.
We talked about whether to discuss particular products and what the products were able to do and realized that regardless of which product any of us are using to do some of these things there are some pretty interesting and amazing use cases that are a little bit platform-independent. And so, what I’d like to do is just go down the row and ask each of you to highlight two of your favorite use cases and tell us a little bit about how that how that plays out and what kind of value it adds.
And let’s go in the opposite direction here. So Sasha, I’m gonna call on you first.
[SASHA RAO]
Oh, I didn’t expect that. I thought I only got on this panel because of my height.
(audience laughter)
Uh, so I think I’m gonna take someone else’s example. I use it a lot for prior art claim charting, so like invalidity claim charts that are due and there’s this time pressure on them, and there are some really good tools where you can throw in your prior art, your claim language, and it creates a pretty good chart showing why the patent is invalid or the claims you are talking about.
So, that’s a very good use case. Another use I do even though we don’t use AI to draft briefs, what we do use it for is to analyze a brief. So, when we get a brief from the opposing side, we stick it into AI, see what’s coming out of it, and then that’s like some sort of a barometer of what might be a point of attack on the brief.
[SHARIF JACOB]
Yeah. I’m, I’m just gonna add one little nuance to the claim charting. One thing I found impressive about some of these claim charting tools is not only does it go and get all the quotes and put them in the right place, which is an onerous task, you know, but it will sometimes come up with sort of new theories of invalidity. “oh, it’s accusing this function. You were focused on function A, actually it’s accusing function B.” And that one sort of goes through all the elements too.
There’s two potential theories of invalidity. That was not something I expected, you know, when I was using these charts. I thought it was more of the manual task.
And so it can add value in surprising ways. Another thing I think is really useful is when the opposition brief comes in, you put it in, you put in the opening brief too, and you ask it, “Make a list of the arguments that have been responded to. Make a list of the arguments that have not been responded to. Give me your best argument.”
And it’ll go through and it’ll make a little list of those things and you definitely can’t rely on You know, I’ve tried to have it write briefs. I use AI all the time. I’m the person who would, you know, when you’re going through the hand-raising, it it can’t write the brief. It’s kind of garbage when you ask it to do that much work all at once.
But it can add some really insightful creative angles you might not have thought about. It can help with organization. Another thing I think it’s really useful for is oral argument prep.
You put it in and you’ve got some ideas of where the judge is gonna go. You give it some of the questions you think the judge is gonna ask and you ask for more questions maybe in certain areas. And it’s pretty good.
Like, it’ll ask the questions that the judge ends up asking the next day. So, I think those are some examples of where it can add a ton of value.
[DAVID LISSON]
Okay. Eric?
[ERIC LIN]
The first use case which I think probably AI is probably where we’re gonna see more of this, is analyzing portfolios at scale. And so that’s during usually pre-litigation diligence. And so that’s diligence work in support of licensing transactions, general IP strategy like competitive landscaping and then also identifying opportunities for enforcement and monetization.
So, what this means is being able to take a hundred patents, be able to auto-classify them based on subject matter, filter by the desired subject matter, like let’s say I want to only look at semi-conductor packaging patents. Take those, let’s say 50 of the 100 is semi-conductor packaging patents, putting that through and identifying targets for potential licensing infringement through the identification of products, through the searching of publicly available documentation that could show the structure and operation of those products and be able to identify at a high level what the risk is and which patents kind of rise to the top. So, I’ve personally seen at least four major law firms try to leverage this technology and at least one of them or two of them have actually won new matters,uh, because we are actually seeing a lot of companies, especially in Asia, who are divesting a lot of their patents and figuring out how to license and monetize, and maybe potentially enforce. So, I think pre-litigation diligence just generally provides a lot of opportunities, and it’s not just for pre-suit investigation or enforcement.
Think about it in a way in which, very competitive and contentious space like medical technology, like medical devices where like yesterday emerging medical device companies are building this kind of protective moat around emerging technologies, and then there’s incumbent players who also have their own IP. Uh, you know, I’ve personally worked on a lot of those types of diligences, and those types of diligences often involve, like, “Hey, if they come at me for enforcement or for some type of licensing opportunities, what can I throw back at them?” Like, are their patents actually good in terms of validity or infringement? And also, what patents in my portfolio could I say, “Hey, you’re saying I’m infringing?
Like, you have some issues with XYZ patents.” So I think those types of use cases are really the type of areas where I’ve seen a lot of growth in terms of pr– uh, diligences at scale.
[DAVID LISSON]
I’m last, so some of my favorites have been taken already.
(audience laughing)
But that’s okay. I do think it’s really great for invalidity charting portfolio mining going out into on the internet or other publicly available sources and trying to get evidence of infringement.
Very good for that. This is the wrong room for it but it’s good on the prosecution side sort of managing or the sort of invention disclosure process. So, I’ve heard of uses where they can sort of monitor their engineers’ Slack channels and pick up potential ideas and flag those as potential inventions.
Kind of think, forward and thinking a little bit bigger picture, trying to sort of think about what all of the, sort of what I think are the best use cases have in common, and it seems to me be about large data sets and the larger the data and the least amount of creative thought that needs to go in. I still think lawyers have a role, a very important role, and we’ll talk about potentially some of the weaknesses of AI and what it can’t do, or at least not yet. But if you’re looking for the sweet spot it seems to be the most data, most routine sort of task. That’s what it’s really good at, and so that’s kind of where I’m looking in the future.
[CHRIS MAMMEN]
Great. I’ll add a couple of my favorites as well. One is, you know, Sharif mentioned a couple of different places in the brief writing, argument prep process.
Someplace else where I’ve found it really valuable is when I’ve got the brief essentially done, and before I file it I’ll run it through and I’ll say, you know, “Here, here’s our draft brief. We represent party X. What are the, you know, particularly well-made arguments? Are there any gaps in the argumentation or any weak spots that you think need to be shored up?” And, you know, half the time, I will disagree with what it comes up with, but they’re credible suggestions.
The other half of the time it’s like, ‘Oh, yeah. Yeah, that’s a good idea. I’ll beef that up a little bit.’ The other one is preparing for expert depositions.
I had a case not too long ago where the CV for the technical expert on the other side ran on for about 200 pages of patents and publications and very technical publications. And you know, there was no client in the world that would pay to have anybody on our team sit down and read all of those articles to see if anything was actually even close to on point beyond sort of eyeballing the titles of the articles. But they’re all available online, so you drop that resume in, and you drop the, you know, the patent or patents in suit into the tool and then say, “Which of these publications are in the same ballpark and why?” And, are, you know, “Has this person taken different positions on some of these issues?” And, you know, it’s a great way to neutralize the glazing-over effect of having a 200-page resume.
Something else we talked about during our initial meetings is, it’s not just use cases that we as outside counsel use, but we’ve all experienced, I think our clients making use of AI as sort of shadow counsel, and does anybody want to share some stories and cautionary tales around that?
[DAVID LISSON]
Well, I can share a story from just last week where, you know, oftentimes the client will send a summary that they had generated. In this case, it was a brief, and they put it into their AI tool and said, “Here, this is it.”
(laughs)
So you know, that in a way, I thought, “Well, okay, that’s kind of my job, but that’s alright. And I took a look at it, and I thought, “This is pretty good.
It’s got the basic facts” But now’s my opportunity to show you know, my value add, right? Because it’s, it can’t quite do what we do. And so that actually was an interesting it short-circuited some of the back and forth because often when you’re with a client, you have to say, you know, “Here’s the brief.”
They said this. Blah, blah, blah.”
Instead, we got right to it, right? We got to you know, as a human lawyer, this is what I think and where the weaknesses are and what they, you know, some of the nuances that were missing.
But, and you got, you skip the initial part. So it was pretty interesting that they thought they were done.
Right? They’re like, “Oh, you know, we know the answer. AI told us the answer.” Well, not quite.
I’m glad we got through some of the initial introductory stuff, but here’s the really. And that’s happened a few times. I would say the thing that scares me the most though as a litigator is the non-lawyers treating AI as their lawyer, like at the client, right? They don’t go in-house counsel.
They don’t go outside counsel. They go straight to ChatGPT or something, and then that’s a problem we have to fix. That’s is happening more and more often, and that’s probably a subject for another panel about counseling your client’s use of AI tools, but happens a lot.
[MATT]
Ends up increasing the legal fees.
[DAVID LISSON]
That’s right.
[MATT]
Yeah. Anybody else have any stories they wanna tell?
[ERIC LIN]
Yeah, I got two examples here. So one is actually not specifically like in-house use, but it’s actually like reverse shadow counseling.
So I had a major IP law firm give me the example of the client handed them 30 prior references on a Friday and said, “We think these are good prior references. They may not be in your initial and related contentions and so let’s try to figure out if we need to amend our related contentions by Monday.” And firm figured out, A, “How are we gonna do this? How are we gonna please the client?
How are we gonna figure this out?” And they were able to do that by leveraging claim charting tools like ours. And that was an example where they were able to, I think, from my understanding, is push back on these being good references by saying, you know, “There’s many limitations that are missing. They’re not as good as my primary prior references and I was able to do it within the span of a weekend and with the use of AI.”
The second thing is a specific in-house use case that I’ve seen a lot more of these days is, especially in the auto space, the in-house department’s wanting to respond to licensor or MPA letters with more substance than just, “Patents don’t infringe and they’re invalid.” And so obviously there’s strategies here about not revealing your best prior references, but what’s been effective is being able to do that analysis in house, do more in house, and then to probably just identify a couple references. And just by identifying a couple references and some arguments that often pushes the attention away from that company to perhaps someone else who isn’t responding or provided some kind of conclusory statement.
So, I don’t know if you, any of you have seen that, but I’m seeing that a lot with specific industries that are being targeted for licensing enforcement.
[MATT]
Anything else?
[SHARIF JACOB]
Yeah, sure. I mean, one of the experiences I’ve always had for the whole time I’ve been litigating is the dreaded memo, right? Nobody wants a big memo. And sometimes you have to answer the legal question.
It’s really important. And I think my experience is that the clients are gonna try that first.
They’ll use ChatGPT, they’ll see if they can figure out the legal answer, and then if they can, they’re not gonna bug you with that. I definitely am seeing clients reviewing our work using whatever, you know, solution they’re using and sending often sometimes the questions, you know, literally directly from whatever chatbot they’re using. We’ll get those.
I definitely–
[DAVID LISSON]
Not just our work, our bills too.
[SHARIF JACOB]
The bills too. So everyone knows that.
On the They get reviewed by AI. Yeah. On the other use that I’ve definitely seen is definitely on the plaintiff’s side.
On the plaintiff’s side, you know, how do you sort through all of these patents and figure out which ones, I think I’m seeing clients are, you know, putting a huge volume of patents in and trying to, at least as a first cut and the work that comes out of that is directional. It definitely needs vetting, but it’s not wrong directional, you know?
It’s right directional and it just needs really careful vetting before you can go live with it. Okay.
[CHRIS MAMMEN]
Anything else?
[SASHA RAO]
I have a, on this point about using AI to pick your patents to assert. I found that it doesn’t pick the right patents. For example, it doesn’t know the law of marking and doesn’t know the effect of having a patent with apparatus claims asserted when the patent owner did not mark. And so you have a potential damages problem especially if the patents are close to expiration.
So I think there’s a lot of people who just use it to send charts out, but a lot of them are just garbage.
[CHRIS MAMMEN]
So I’m gonna shift gears here a little bit. One of the key places that everybody in popular media hears about the use of generative AI in law is when hallucinated case citations make their way into briefs that get filed somewhere. And that has resulted in a large number of courts around the country issuing various kinds of guidance on this topic.
And Sherree, if you wanna tell us a little bit about some of the key jurisdictions on that?
[SHARIF JACOB]
Yeah, absolutely. So, I think, suffice it to say, we’re all using AI, but you really, you can’t do it for every case. And every, not every judge yet, but many judges in the jurisdictions in which we commonly practice patent litigation have guidance on their website in the form of order that applies to you, you know, the moment you appear in that litigation.
So, it’s pretty important to understand what those orders say and what the sort of issues are that get raised by those orders. Here in the Northern District of California, we’ve got Judges Martinez Olguin Judge Lin, Judge Lee, Judge Kong, they all have orders on the use of generative AI. And those orders fall into two different buckets.
One order is, “Okay, you can use AI, that’s fine, but you have duties of competence and you’ve gotta meet Rule 11, so, you know, sort of beware.” That’s one sort of order. But there’s another order in the Northern District that Judge Martinez Olguin and Judge Kong both have, which says, “If you use AI to write any portion of a brief or to otherwise contribute to a brief, lead counsel has to personally check the citations.”
So, that’s a little bit different, right? And you know, something you just need to be aware of. The Central District of California also has orders that are more sort of on the lighter side of the Northern District of California. The Eastern District of Texas has a local rule about the use of AI, and basically it says that if you’re gonna use AI in a case, you have to meet the requirements of Rule 11.
The Northern District of Texas also has an order, and in that order, you have to certify one of two things. Either that you’re not gonna use AI, or if you’re gonna use AI, you’re gonna look at every single citation, either using a book, like a book, or you’re gonna use Lexis or Westlaw or one of the others to check all the citations. And you have to certify that before when you appear.
You have to sign a certification when you appear. I think the other area you sort of have to be careful about for kind of secret traps is protective orders.
We have been drafting protective orders for another era. And a lot of protective orders out there will allow you to use, you know, a sandboxed confidential AI setup to do work. Oftentimes, that would fall under the vendor bucket. But you have to look closely at the protective orders and the e-discovery order.
Some judges have sanctioned parties for using AI in violation of the protective order. For example, some protective orders say your vendor has to sign the attestation that comes with the protective order. And, you know, are you doing that?
Did you get ChatGPT, you know, to sign the attestation? So you really have to look at the specific protective order, you have to look at that judge, and you have to figure out, does it apply? I think more and more firms, including mine, are starting to negotiate with opposing counsel a provision in the protective order that specifically governs AI. And it’ll often say something like, “Nobody can use AI unless it meets these criteria.
It’s confidential, sandboxed, blah, blah, blah.” And, you know, so far, in limited experience, but so far, counsel has been open to that and willing to do that because of how ubiquitous the use is.
[CHRIS MAMMEN]
Yeah. So one court that I’ve come across, ’cause my firm has a substantial office in the, western District of North Carolina, is I think possibly the most locked down version of AI order. And what it requires, all of the judges in the district have signed onto this. Is that every brief that gets filed in every case in the Western District of North Carolina has to include a certification that no AI was used in doing research for that filing except through Westlaw, Lexis, FastCase and Bloomberg.
And so you know, it’s really drives home the theme that Sherree identified, which is, you know, you learn as a junior lawyer, you gotta check the rules, check the local rules, check the judge’s orders. That is extremely true in this area where, you know, maybe it’s a judge that says, “Hey, you’re on your own.
Follow the rule, follow the otherwise applicable rules, or file a certification once, or attach something to every single thing you file in my court”. So there’s some real concerns there. For those who are keeping score at home, there’s a hallucinated case citation database that is reporting out all of the reported decisions, where hallucinated case citations have resulted in orders.
And there’s, when I checked it a week or so ago, there were about 500 cases in the US and another 100 or so around the world. And the kinds of court rules that Sherif mentioned in the Western District of North Carolina. There’s a handy tracker that Law360 has published that keeps track of those. So it’s a good way to just double-check on that.
All right. Let’s pivot now.
We’ve talked a little bit about in the context of filings, ethical duties. So let’s do a little bit deeper dive into those. Are we stealing your thunder?
We’re doing that panel. It’s all good. Oh, okay. I’m just gonna repeat it.
Okay. So we’ll just touch on this briefly, and then leave some of it for you. So there’s really sort of four or five areas where the ethical rules come into play.
Any of our panelists wanna chime in on some of these?
[SHARIF JACOB]
I’ll just chime in very briefly. And I’ll save the details for you guys.
So the duty of competence, you have to know what you’re doing. You can’t do sloppy legal work, no hallucinations.
The duty of diligence, effectively the same. The other thing about the duty of competence is you have to understand the tools that you’re using.
That’s required, you know, by the various jurisdictions. And they’ll talk a little bit more about how that ends up manifesting. The other thing you have to do is you have to supervise people who are using it. If you’re a partner at a law firm, you need to have a policy that governs what the associates are doing.
Attorney-client privilege and attorney work product, you can’t waive it by just putting a bunch of information into a chatbot that isn’t sandboxed and isn’t carefully protected. And I mentioned the policies as part of the supervision. It, the, both, there are two model rules that you should know about, and ABA has a model rule, California has a very similar model rule. They’ll go into some of the, more of the details on those.
But there is some guidance out there now about using AI and pretty important to read it and be familiar with it.
[CHRIS MAMMEN]
Yeah.
[SHARIF JACOB]
And can I just note another set of, I’m gonna call them rules? Really, really check your client engagement agreements, especially with big companies. And some of them will deal with it expressly.
Most of them, in my experience, have not yet. Those that have dealt with AI expressly often say, ‘Don’t use AI,’ despite the fact that their legal department wants you to. You know, the clients you’re dealing with probably want you to use AI. Oftentimes, the company or the engagement agreement says don’t.
If it’s not expressed though, there are a lot of other provisions that come into play that you have to take a look at, including who owns the data, if you’re trying to run, if you’re building your own AI tool or running things through your own system, there’s a question about whose data are you using. Is that yours? If you’re a big firm that has a lot of precedents, are those your precedents?
Are they the client’s precedents? There’s a lot of stuff in the engagement, in the fine print of the engagement agreements, that you have to look at carefully, both when you’re using AI for a particular client or when you’re trying to build out AI tools for general use.
It is, in my experience, that comes into play more often than the more general ethical rules that I think we’re all pretty comfortable with from our day-to-day practice, you know, without AI.
[CHRIS MAMMEN]
Anybody have anything else to add about counsel guidelines and client guidance? I’ve got a couple of comments if, so I think you know, the life cycle of outside counsel guidelines are often much longer. They’re, you know, they get put in place and they get updated every couple of years or less frequently. And a lot of a lot of AI prohibitions were added to outside counsel guidelines sometime between November of 2022 and July of 2023, when there wasn’t a lot of enterprise-grade protection on AI platforms and there were a lot of concerns about protection of client confidential information.
And when the AI products and platforms were largely standalone as opposed to incorporated features in all of the other tools that we’re using. And so, you know, the outright prohibitions that show up in a lot of outside counsel guidelines or the notice and specific product-by-product consent types of guidelines can be really challenging for, for a law firm to comply with ’cause a lot of times, you know, good Lord, how many different AI features are there now in Microsoft Office?
And, you know, do we need to disclose all of those and get permission, for example?
[DAVID LISSON]
Well, and I don’t know if anyone else has had this experience where your little C client, right, the individual you’re working with on a litigation, and your big C client, as defined by the outside counsel guidelines, don’t always agree on how much you should use AI, right, because the oftentimes, the in-house legal department wants to keep costs down, wants to be, you know, figure out how to use it, and the outside counsel guidelines were drafted, to your point, two or three years ago, and that, you know, that causes some tension and no one really wants to get into renegotiating outside counsel guidelines, and your little C client, the person you’re working with, doesn’t want to fight, you know, City Hall. But that is something that happens quite regularly.
[CHRIS MAMMEN]
So I think just one comment to sort of close out on this area and it’s where we started on this, was the duty of confidence includes a duty to be aware of both the risks and the benefits of available technologies and, you know, there’s a debate to be had about whether it includes an affirmative duty to adopt some of these technologies in order to deliver legal services better, faster, and cheaper and to be, stay abreast of changes in the law around that as well as changes in technology.
[ERIC LIN]
And I think–
[CHRIS MAMMEN]
Yeah.
[ERIC LIN]
That’s really important in terms of understanding what the strengths, weaknesses, and limitations are because I think a lot of people have the grand notion that you type a simple prompt in, and voila, you’re gonna get cancer, cancer’s cured, right? And in the legal context that’s especially important where everything is about precision, accuracy, efficacy, and so what I think where we’re seeing a lot of quote-unquote ‘issues’ is the whole reality versus expectations portion, but once you’re able to direct and have the proper expectations and leverage it in the right way, I think it’s definitely a value-add, and we’ve seen more tech-forward, advanced practitioners talk about how they are using AI has saved efficiency and been able to generate a RIO, right? But at the same time, we also see the same practitioners who are very grounded in our ways, right?
I think most of us, if not all, we went to law school, we went to undergrad without any kind of AI, let alone generative AI tools. I mean, ChatGPT was only released three years ago.
Only three years ago. It feels like six years, at least. But we all have a set routine, and the problem with the set routine is changing it, and I think that whoever is able to kind of switch it out. You know,
(unintelligible)
talk about how he uses AI all the time but not everyone does that. And we are all accustomed to that and I think understanding limitations and just switching a little bit, that helps with getting the proper value versus what you think it should do which is like huge expectations.
[CHRIS MAMMEN]
Next question I wanna turn to is one about how do we decide which products are okay to use within our firm or to deliver client services? You know, is this something where we as individual lawyers need to be reading down into the disclosure and use policies of the various platforms we wanna use or are there some emerging best practices around that?
Do any of you have experience around that?
[SHARIF JACOB]
Well, I’ll just start with the ethical requirements. They’re actually on ethical requirements in the ABA opinion, and in the California Bar opinion about the adoption of AI, and you have to know and understand the tool and its limitations and its impact on your clients before you adopt it, and you can do that either by self-study, that’s what they call it, which is you figure all this stuff out for yourself, or if not, you need to either associate with people who have the expertise or alternatively, you need to have vendors and employees who have the expertise. So, the bottom line is, you know, the ethical guidance says that you probably need to have the lawyers and IT working together, and you do actually have to read all the terms of use.
The ethical guidelines are quite clear about that. So, you know both ABA and California lay out a process that these guidelines are, you know, considered recommended. They’re not necessarily mandatory, but I think it’s probably safer to treat them as mandatory.
[DAVID LISSON]
And in this respect, I don’t, I’m not sure if we’re that different than you know, non-law firms and non-lawyers in that when I advise clients about setting up an AI sort of policy and protocol, my first advice piece of advice is set up a committee with stakeholders that include both the AI evangelists and the IT folks and the data privacy folks. And our, you know, we have the GC’s office that knows the outside counsel guidelines, because it’s really too much for one individual lawyer or engineer if you’re, you know, at a different type of company, to make that decision on their own. You also lose a lot of knowledge and experience if you’re just focused on your own stuff. As I said earlier, one of the benefits of a big firm is you can set up different sort of experiments and trials, and you want all that info, all the good and the bad filtered up to a committee.
So we have a committee that does that and it’s got lawyers on it. It’s got IT folks, data privacy folks, as I said, the GC’s office. There’s just too many different considerations to bring to bear, but I do also think it’s really important to have the heavy users on that committee because you want, and for us, that’s often the associates who are thinking about this a lot more nimbly than us old folks are.
And so you know, all of that gets fed into the committee.
[ERIC LIN]
Good. Yeah, and having– having been on both sides now I, I left law firm practice about nine months ago, and now I’m on the receiving end of a lot of questions or lack of questions from both companies and law firms. I think there are probably two or three questions that I think are most important in terms of how a law firm or company should evaluate, because not everything is created equal. And so number one is security. I think security is still critical.
You would, you shouldn’t assume that every single company has the right safeguards, guardrails, the right technical architecture, and let alone, like certifications. You’ve heard all these things from SOC 2 or ISO. Those certifications are from auditors, and there’s many different types of auditors who are just willing to hand out these certifications.
And so just because someone says they’re SOC 2 certified doesn’t mean that they have the right necessary ways to establish security. But the good thing about patent work, and it’s often good to make Info Sec teams aware of this, is a lot of patent work is public, right?
Patents issued, patents published, patent applications, prior art references, publicly available evidence of use. That’s all public, and there’s less of a risk in terms of security. But obviously if you’re thinking about ever connecting this to a discovery database, that’s definitely a big thing. I think, and then the second thing besides security is like how integrated is this into my workflow?
And this goes to my point about how there are a lot of point solutions out there, but like how well does this, is this integrated in terms of not only being able to do claim charting, but also being able to provide the prior art search results, be able to filter down, and then be able to map limitations in view of the claim construction and change that claim construction to reflect different mappings. And then finally lead to, for example, invalidity contentions that are in contention format with standard boilerplate language. So like, think about all the different steps because as for lawyers like us, it’s hard for us to divert ourselves to, once you’re, once you’ve done one, step one, two, three and be like, “Oh, step four is where we have some type of solution.”
I have to divert myself there and then return back to like five, six, seven, how technology can assist with every step of that process.
[MATT]
So with our about seven minutes left let’s turn to the future. Where is this taking us all? What is this gonna do to the practice of law and and the business of law? Um, Sasha, you wanna–
[SASHA RAO]
Yeah.
[MATT]
–lead off?
[SASHA RAO]
I think this is gonna be a massive shift in the labor market generally and specifically for lawyers. I see it already with some of the big clients. They’re moving a lot of the legal services they typically used to hire lawyers for to machines. Now I’m talking about, you know, basic contracting functions, marking up non-disclosure agreements, doing deal, deal diligence.
Look at all the IP in this company we’re about to acquire. You know, is it worth anything?
That’s all being done inside the company with machines. And so think of all the jobs that are going to be lost in the legal profession. Now, Matt, your job’s not gonna be lost because the trial lawyer’s always gonna be there. You’re fine.
But, you know, for the lawyers starting out who haven’t yet chosen their art form, what does the future hold? And I think it’s somewhat bleak, right? You know, in the old days, you read all the documents, you knew your stuff, you know your documents and your cases well. But, you know, the future lawyers are not gonna be trained in the same way.
And so I don’t feel, like I feel like we could all lose our jobs in five years, right? So– Matt.
[MATT]
All right.
[CHRIS MAMMEN]
Yeah.
[AUDIENCE MEMBER]
So I commend for everybody, and this is a great panel and it’s hugely important. The Sedona Conference is doing a project, it has a new working group all on AI.
If you’re interested in trying to promote thought leadership in this area in a variety of ways, I commend it to you. But one particular effort that we’re just starting now is to think through the impact of this on legal education as a way of both inputs and outputs of training people to be better users of AI so that they’re using the tool effectively and understanding it.
Law schools are all over the map on this. Some of them prohibit it in theory, others are trying to get ahead of it. But also, giving some comfort about what their, the proper role of this is. How should, what do young lawyers to be think about this impact on their career?
Either as a plus or maybe as a sign they should be doing something else.
[CHRIS MAMMEN]
Yeah.
[AUDIENCE MEMBER]
So if you’re interested, I can really commend that effort because it’s something that I think is gonna be important to do fairly quickly because the impact on the profession is huge.
[CHRIS MAMMEN]
Thanks. Thanks, and for the recording the comment was about the Sedona Working Group on AI that’s forming. A lot of urgent work there, particularly as it relates to legal education and how we incorporate this into training our next generation of lawyers.
[SHARIF JACOB]
Yeah.
[CHRIS MAMMEN]
Sharif, do you have a, you know?
[SHARIF JACOB]
Yeah. I’m a little bit less doom and gloom than Sasha is. I think we’re living through another industrial revolution, and I do think it’s gonna change a lot of things. I don’t think it’s gonna make lawyers’ jobs go away.
I think we still have to be in court. We still have to do deep thinking.
I think it’s gonna take away some of the more onerous, you know, parts of it. When I was an associate, I remember spending, you know, 40 hours doing a claim chart by hand. Like, who wants to do that? I’m not mourning the fact that our associates don’t have to do that.
I think it’s gonna make us more efficient. I think it’s gonna make us more nimble.
I think it’s gonna make us able to do more work in a shorter amount of time. Yes, will it make me doing one case cheaper and faster?
Yes, but then I can do more cases. So, I really don’t see it as sort of obliterating the legal profession. I do think, though, that you need to use how to use AI, and I think that, you know, it’s gonna continue to advance and the uses are gonna get more complex and sophisticated. And what I’ve noticed, somebody said some– sometimes the younger users are better at using it.
I also sometimes see new attorneys who are, like, resistant to it ’cause they’re mad at it. You know, they’re mad that it’s coming for their jobs, and I don’t think that attitude is sustainable. I think that’s like standing on the beach and the tsunami’s coming, you know, and you’re sort of putting up your middle finger.
I don’t think that’s gonna get you very far. You know, I think you need to adapt.
All right, Eric, one minute.
[ERIC LIN]
Yeah. Uh, so I was talking to two Chief Legal Officers who are advisors to Patlytics. One is Paul Grewal Chief Legal Officer of Coinbase, the other is Amar Mehta, Chief Legal Officer of Waymo, and both said, “Do more with less.”
And that kind of echoes what everyone is saying here, which means for in-house counsel, what they both said is they think that there’s probably gonna be a stabilization of in-house roles, but that in-house folks who know how to leverage technology will have stickier roles because they’re gonna be able to do, I don’t know, five people’s jobs. And then for outside counsel, as folks have already said, a lot of the top trial lawyers here, luminaries of the profession, your strategy, your argument, that’s always gonna be there, and that’s what they’re willing to pay the hourly rates, whatever, $2,000 an hour for.
It’s always gonna be there, but more of the grunt work, which could be more than 50% of the matter, that’s where, maybe in like three to five years you’re gonna see that become more fixed fee, maybe discount, heavily discounted rates. So, I just wanna impart that because the common thread of all this is the whole do more with less.
[SHARIF JACOB]
David.
[DAVID LISSON]
All right. Um, well, I’ll be quick.
Uh, I, I’m, I think their jobs will always be here. They will change though, and that’s kind of the challenge. I’ve been doing this long enough, I’m not, it’s only been 20 years, but in this, the last 20 years I think I’ve heard that associates jobs are gonna go away two or three different times with various things, right? First it was e-Discovery instead of boxes and then it was the way you search was gonna end doc review and all this sort of stuff, and it didn’t.
It, things changed. And so, that I think is what we have to manage, and one of the tricks for, as I’ve gotten more senior, I learned how to review an invalidity chart by spending many hours crunching through invalidity charts and figuring out what made a good one and a bad one. I learned how to write briefs by first writing stupid discovery letters where, right?
That’s how I learned how to be an advocate in writing as a first year. And so now, I think our challenge is how do we still teach those lessons so that folks can become senior lawyers that can do the value add that lawyers have to do without the grunt work? And that’s my big challenge.
[MATT]
And that’s a, that’s a great place to end is, is you know without going through the, the Karate Kid way of learning it how do we train the next generation to be the next Matt Powers? Quick practical question. Has any one of you found a good tool to write not invalidity but validity charts? Maybe something to discuss during the break.
[DAVID LISSON]
Yeah.
[MATT]
All right. Thanks everybody.
(applause)