The Future Of AI and Its Impact On Legal Professionals Efficiency With Damien Riel

Made with speech-to-text software, so a few words may be off.

0:00You're listening to the Contract Heroes Podcast, your one-stop shop for all things contract management. And now here are your hosts, Marc and Pepe. All right, guys, so thanks a lot for the people that came this early in the morning for Central Time. For Eastern Time, that not, not, not so early. So, uh, this is going to be one of our... We, we, we got a couple of episodes for, for Contract Heroes doing, uh, live streaming in this. Today we got Damien Riehl. For people that don't know you, Damien, like, can you give us a little bit of background? Who is Damien? What do you do? Without talking about professional life, more like what kind of music do you listen to? That would be cool. I'd go at- Just to make a change before talking into that techy and geeks type. Yeah. So, uh, I've been a lawyer since 2002, but I've been a musician since age 10. So, uh, here you can see my guitars, you can see my drum set, you can see my piano. So I play music, uh, I sing a lot.

0:59I was a voice and percussion major in college, uh- Oh, wow... and then, uh, uh, uh, graduated with a degree in, uh, vocal and, uh, instrumental music. Uh, and I was conducting a Brahms piece and two of my tenors started punching each other in the face. Uh, so I decided maybe I should go to law school, so then I did. That was a good choice. Not, maybe not as fun as becoming a musician, but it's, it's, it, it's always good. And, and- So my, my, my voice teacher said, uh, he, he was an Austrian guy, he sang in the Vienna Boys Choir- ... and he said, uh, "What are you going to do? Sing to the jailbirds?" And, uh, I'd like to say that I, I used to break up fights in the f- in the c- classroom, now I break up fights in the courtroom. That's awesome. So, uh, well, guys, as you know, there's a lot of buzz in the AI, you know? So I think one of the main things, and one of the main reasons why I thought this conversation would be really interesting for a lot of people, Damien, is because since ChatGPT, you know, uh, 2022, November, it's, like, past and post, pre- and post-ChatGPT, and that was just, like, a revolution and people were just like, you know, there's a lot of investing, there, there's a lot of new companies, there's a lot of, you know, maybe, and a little bit, a, a little bit of confusion on how you can use AI properly because, you know, we've seen a lot of different uses for designing, for, you know, copywriting.

2:22But for the legal profession, as you know, we are not very tech-savvy. We're, we're not famous because lawyers are, are very tech-savvy. So I want to break a lot of those myths, right? Because I've seen, like, a lot of people using AI wrong, like asking direct questions like, "Hey, can you give me some cases on how I can, you know, give some of the grounds for my argument in a courthouse?" Or whatever. And, and of course they are going to get wrong answers. So I think we can talk about how... Well, first of all, like you, you can tell us, like, how you got in this industry, right? That would be interesting. What, what have you seen? And then there's a lot that we also want to talk about regarding SALI. If you haven't heard about it, we're going to get a little bit deeper into that, uh, further on. But let's, let's start with that, Damien. Like, can you tell me, in your experience, like how you get involved into AI in the legal practice? That would be, like, uh, really interesting to talk about. Sure.

3:20It, it starts at age 10, literally. In 1985, I was b- coding Basic, uh, in my bedroom. And so I, I've loved coding since 1985, started coding for the web since 1995, then I went to law school in 1999. So, um, I, I came with this at, as a coding mindset even back in 1999 when I started law school, and then for 15 years, uh, I litigated and I realized the practice of law could be so much better, uh, than it currently is. There's so much drudgery that goes on in the law. Uh, I was just at a conference with Casey Flaherty, who your audience probably knows. Yeah. Uh, so Casey just, uh, he made a really good comment that I'd not thought before, uh, that he said this, uh, we do so much drudgery that it's, uh, he said we should be doing more thinking and less thunking. Uh, and I think that that, when, that's something I'd noticed as I was a lawyer. There was so much thunking. There was so much just awfulness, uh, that could be made better. So I made, I pitched Thomson Reuters in 2015. I said, "Here's legal tech that can change the world. You just build it and hire me." And they were stupid enough to hire me.

4:15So I, I l- helped lead a team of 100 programmers and 50 lawyers building this really big thing. Um, since then I, I did cybersecurity for a while. Facebook hired me and my company to investigate Cambridge Analytica, so I spent a year of my life on Facebook's campus. I did that for a while. And then, uh, currently I, I have a data set that's a dream data set of a billion legal documents, with a B, uh, across 100 countries worldwide that I'm running large language models across and, uh, being able to do cool things. That's awesome. And, uh, well, yeah, I mean, those cases sound, sounds super cool, like you being like even maybe in documentaries without knowing them. I'm gonna, next time I'm gonna watch the Cambridge Analytica and maybe you'll be there. Well, so I, I, I've watched that and yeah, there's a lot of familiarity there. Yeah, that sounds so cool. So one of the things that I also wanna talk about, I think it would be like really good to set the grounds on how AI can be used properly is let's talk about SALI, the SALI Alliance. And can you explain like how SALI standardize... Well, first of all, what is SALI? Mm-hmm. And then we can talk about what it does. Sure. Uh, SALI is a legal data standard.

5:24So think, people know ISO, uh, that if, you know, if you're in the United States you can plug your, uh, plug your electronics in anywhere because there's a standard there that says this is what a standard looks like, uh, for electrical things. This is what the standard looks like for all sorts of things. So we are a legal data standard that's being used by Thomson Reuters, by LexisNexis, by iManage, by NetDocuments, by Microsoft, by their in-house counsel, Intel, by their in-house counsel, as well as by all the biggest law firms in the world. And so what we're allowing is for the Thomson Reuters and the iManages of the world to be able to talk to the law firms of the world, to be able to talk to their clients in the world in a standardized way. So the things that we're counting are things like area of law. Uh, everyone is counting area of law, but everyone has a different name for the areas of law until SALI came along, and now SALI has the same identifier for, uh, the area of law that now I can push and pull data through standardized API calls to be able to push and pull data through them. That's the, the high level overview. That's really cool. And I'm actually, for the people that are interested to learning a little bit more about SALI, I just put the website on the comment section. It's www.sali.org.

6:34Yeah. S- S-A-L-I, yes. S-A-L... Yeah, I'm sorry. Yeah, yeah. No, 'cause it's often confused with S-A-L-L-Y, uh, which is the woman's name. It's just one L. A, a, one L and then an I, yeah. And okay, that's really cool because now you can see, like, a lot of really big companies, big, uh, law firms are sharing information, and that's... And I would say that that's really useful because, I mean, useful and, and I was really impressed at the beginning because wh- you know, when I was at, working at the big law it's, you know, you cannot, like, share that type of information because it was maybe considered, like, intellectual property of the law firm, you know? And this is really cool because- Yeah... there, it's like a huge database of information. And so you can help everybody to, you know, use some more standardized terms and, you know, maybe help them with their arguments, legal opinions, or even contracts, and I really love that part. And when it comes to AI, Damien, like, how do you think this type... Having all the database, you talk about one billion, yes, with a B, documents, and how can you use AI to streamline that part of the legal research?

7:49And what do you think it makes them stand out from the, you know, how people are used to do the legal research? You know, maybe just going by the books or- Yeah... going to the library. It, it might make, uh, sense for me to go through how you could be able to tag up your data with SALI, then, then from once you people figure out how to tag up the data in SALI, then how to use the AI to do, to go through the tag up. Yes. Uh, so maybe, um, uh, if it makes sense, I'll share my screen and we can be able to- For sure... we can talk through that. Yeah. So with, uh, with SALI, uh, most importantly, SALI's a nonprofit, uh, and I'm a volunteer for that nonprofit. Uh, everything we do is free and open source. Uh, we are applying for ISO standards, so we will be an ISO standard soon. Um, everything is free as in speech, free as in beer. You could be able to download it today, and, uh, you don't have to pay a dime, and you can extend it too. Uh, everything's on GitHub. Um, here are some of the, you know, small companies that are using SALI, uh, like Thomson Reuters, Lexis, Bloomberg, NetDocuments, and iManage, Microsoft, Intel, GlaxoSmithKline, and the biggest law firms in the world, all of which are coalescing around SALI as a legal data standard.

8:51Uh, for each of them, we're solving an internal problem and an external problem. The internal problem that Thomson Reuters has is, uh, they said, "Hey, we have 30 products and they don't talk to each other. Make them talk to each other." And then they looked at SALI and they said, "Oh, they have everything I need." So now they're using SALI internally to be able to connect all 30 of their products. Then they're solving an external problem, uh, to be able to say, "Hey, law firms, if you wanna talk to these 30 products, send me the SALI tags, and then you can talk to all of my products." Uh, and so this is, uh, the same internal and external problem that we're solving for every single one of these, uh, logos here. Uh, everyone has internal systems that don't talk to each other, and everyone has external systems that they want to connect to. Uh, and so SALI is the way that, to be able to interoperate between vendors, law firms, and their clients. Uh, and so this is, uh, this is, interoperability is, is, uh, the important part. So when you think about what is, you know, what are you counting? What are you interoperating?

9:42Uh, everyone has the area of law, that is, uh, things like intellectual property law, uh, and everyone is counting a thing like a patent law. Uh, and of course, you might think, well, there might be children that are patent litigation and patent registration, prosecution. Um- Yeah... but you'll see that these aren't, these are not children. Um, SALI smartly separated the area of law from the service that I as a lawyer provide, because on day one on intake, the client says, "Let's go ahead and f- uh, should I get a patent or not?" That's day one. And then day three they say, "Yeah, let's go ahead and file the patent, license the patent, litigate the patent, and deal with patent assets and bankruptcy." Each of these is a separate tag, uh, that is not a child of this. Because you can imagine- Yeah... if you were to instead have a child, then, uh, you'd be able to say, "Hey, patent litigation and patent prosecution," but then you'd have to do the same thing for a copyright litigation and copyright registration. Same thing for trademark litigations, right? And it's just turtles all the way down. So tag number one is patent law, and tag number two is advice, and then day three you say regulatory, so you keep tagging them up.

10:43And the most important part of SALI is that of course, it's always called SALI, unless you're German or unless you're Spanish or unless you're French or Hebrew or Hindi or Japanese or Chinese. So there are many ways to express the thing that is patent law, but there's only one unique identifier that is patent law. Uh, and that unique identifier is what's being used by every single one of these companies here. So if you were to send me an API call saying, "Damien, send me all of your patent law things," you would send me the SALI API call that has this identifier, and then I would send you all of my patent law things. Change the endpoint to TR and say, "Give me all your patent," and they'll send you eventually all of their patent law things. Lexis, Bloomberg, NetDocuments, iManage, everyone is counting patent law, the concept, with this unique identifier regardless of how that concept is, uh, is actually expressed in language. So this is why we're getting, uh, such traction is because everyone here realizes that this is just a smarter way to be able to, be able to have, uh, represent what the things you're doing.

11:41Um, so another thing that, uh, is negligent misrepresentation. If I say something false about Pepe, you can sue me for negligent misrepresentation, and that's probably going to happen by the end, so, so watch out. Um, negligent misrepresentation- ... if you ask three lawyers what kind of claim is negligent misrepresentation, lawyer one would say it's a negligence claim, obviously, and they'd be right. Uh, but lawyer two would say, "It's, it is, but it's also a misrepresentation claim," and they'd be right. And lawyer three would say, "It's, it's both those things, but it's also a defamation claim," and they'd be right. So how do you settle the argument between lawyer one and lawyer two and lawyer three? And the answer is you don't, because it is a negligence claim. It is also misrepresentation, also defamation. Each of those is a type of claim, and some jurisdictions in the United States call it negligent misrepresentation causing harm. That is a synonym. But all of those have the same unique identifier that is right here, that is negligent misrepresentation. So this is just a smarter way to be able to represent the law as it is. Uh, it is not any one of those things. And similarly, if you were to say a maritime negligence, uh, if you were to get stuck in the Suez Canal, it would never happen, but hypothetically, if you were to get stuck, uh, you would get sued for maritime negligence, which is a negligence claim.

12:46It is also an admiralty and maritime claim, which is argued in admiralty court. You can see that we've built in relationships. Uh, and admiralty court is related to the area of law that is admiralty and maritime law, and is related to the industry that is the transportation and logistics industry. So here you could see, uh, you can imagine a law firm saying, "I'm a partner. I have a new client that's in the transportation industry. What have we done in that area of law?" You can follow this smart graph to say, "We've done maritime negligence and these 20 other things as well." Uh, so this is just a smarter way to be able to represent, uh, both multiple parentage, that it is a negligence claim, it is also an admiralty maritime claim, and then relationships, that these are related to each other, and all of those are reflected in these here. So anyway, so as you think about how, uh, how could I use this in my law firm? How can I use this in my, in my corporation? Um, you could tag up all your documents. So, uh, merger agreement is a SALI tag. Uh, and then within that document, directors and officers is a SALI tag. Exchange agent, corporate organization, uh, regulatory matter, broker fees. Each one of those is a SALI tag.

13:52So what NetDocuments is doing is programmatically saying once you upload your document, they'll be saying, "Hey, we found these 30 SALI tags. Would you like us to tag these things up for you?" And you can say, "Oh, yes, please." And then these are programmatically tagged up for you. So now you could be able to reason with these documents saying, "Show me all the merger agreements that involve broker fees," and then you'll be able to pull it up. So that's the power of tagging up your documents. Uh, similarly with timekeeping, uh, deposition is a SALI tag, contract is a SALI tag, affidavit, statement of undisputed facts. Each one of these is a SALI tag. So you could be able to say, "Show me all the patent case where we had a deposition, uh, where there was a statement of undisputed facts." Now you could be able to filter and better reason with these things. Uh, so anyway, so that's a precursor to how knowledge graphs, which is what SALI is, uh, you can see that this is a graph how things are interconnected and then interoperates with large language models. Because once you tag them up, then you could be able to better say, um, here is the section of the document that involves DMCA claims, which is different than the sections of documents that is the breach of contract claim, which is different than the section on tortious interference, et cetera.

14:58Now you can put spans between the document spans, and now you can see th- you can use these spans for retrieval augmented generation for large language models. So if you have a question that you wanna say, "How is the DMCA claim being processed?" Now I know that I've tagged up DMCA claim, and I've tagged up the span of the p- the part of the document that has DSC- DMCA claims, so now, now I can put that into the large language model and be able to connect that with the hundreds or thousands of other documents that also have the span, and then I can better get an answer because now I've well tagged this thing and I've well tagged the spans of language that relates to that thing. So, um, so anyway, so that's, that's a longer answer to your question, but- ... but, but I would say that, you know, Pablo Arredondo of Casetext and I were co-presenting on a panel on, at Alticon in July of 2022, so right before the explosion of November of '22. Right. But in July of '22, uh, the topic was knowledge graphs like SALI or neural nets like then GPT-3.

15:59And Pablo was on the neural net side and I was on the p- on the knowledge graph side, and I was saying, "Knowledge nets or knowledge graphs," and he was saying, "Neural nets." And I said, "Let's have a celebrity death match between us." Uh, but, but in the end, we actually agreed more than we disagreed, and I think that we need the underlying knowledge graph. That is, we need the underlying logic to say that maritime negligence is a negligence claim and an admiralty and maritime claim. But then once we have that well tagged up data, then the neural nets can more efficiently go through and be able to reason with legal concepts in a way that, uh, without the n- uh, knowledge graph is impossible. So anyway, that's, that's a long, uh, long answer to your short question, but I hope it was helpful. It was really. What do you think, Marty? You're, you're not a lawyer, so I think that would be- ... in- i-interesting, like it's good so that people can know like what do, do lawyers... Some of the struggles when you're doing some legal research. Right. Right. Yeah, and, and I think, Damien, my question is, you know, around how, how will this impact and, and, you know, when we talk about other AI applications as well, the conversations or the intersection of, of outside counsel and inside counsel, especially for, uh, maybe some of those smaller law teams, l- uh, legal teams that are out there?

17:09Yeah. Uh, so, uh, that's a really good question, and, uh, maybe a good, uh, a good name to drop is Jason Barnwell from Microsoft. Uh, Jason, uh, has, uh, has said essentially he's doing what I'm describing and now he's handed the baton to some other folks. But he, of course, is in-house at Microsoft. Uh, and so he has, uh, said that he's working with his law firms. And so what he's doing, uh, for internally, whenever a business person has a legal question, uh, say they send it through Slack or whatever the, the Teams I guess they would use within- Teams ... Microsoft. Yeah. Uh, so if they send a Teams message saying, "Oh, I have this legal question," the business person says, he's tagging up that legal question with SALI tags, uh, to be able to say, "Here's, here's the... It's a question on privacy," or, "It's a question on trademarks," or, "It's a trademark," or whatever, right? So he tags it up internally, and then once he's tagged it up internally, then he's searching through his internal knowledge, uh, to be able to see if he has that answer. And if they don't have that answer, then they kick it to a law firm. Uh, and then they say, "Hey, give me an answer on this thing."

18:06And then he's saying to his law firm, "Go ahead and tag it up with SALI tags as you give me the answer, because now I can be able to get that well tagged data and bring it back into my knowledge graph." So three months from now, when another business person has a similar question, now it could be well tagged to be able to say these are the types of things. Um, so, um, that goes to, uh, you know, the smarter people in the audience might say, "Well, can't a embedding, uh, vector embedding search, uh, be the same kind of thing than tags?" Um, and where te- in vector embeddings, uh, fall down that, uh, that knowledge graphs are actually really good at is things like analytics. So if Bri- if Jason wants to be able to say, you know, "What percentage of our questions are trademark questions?" Or what percentage of our questions are, uh, privacy questions. Um, and LLM is not going to be able to answer that. Uh, so a- analytics, uh, you need to have the tags to be able to do the analytics. And then similarly for interoperability, uh, if you want to take internal systems and make them interoperate, then you need SALI. And if you wanna connect those internal systems to external systems, uh, you need SALI.

19:07Um, LLMs cannot do that kind of work. They cannot... You know, your internal system has, you know, say Llama as a large language model, and the external system has GPT-4, or the external s- this external system has Gemini, right? So if everyone is using different systems, you need the standard. That is, you need the standard tags for the interoperability. So can tags and taxonomies do that LLMs can't? They can't do analytics, and they can't do interoperability. Wow. I mean, I wanna switch this a little bit because we, we talk about a lot of how you can use it for, like, let's say, getting an argument for a trial case or, uh, you know, legal research. But since the podcast is called Contract Heroes, like, can you tell us any example on, on how, uh, lawyers or people on the commercial teams dealing with a lot of contract can use this type of AI tools to maybe understand better the contracts, or maybe for negotiation part, or any of the phases into the contract management process? Sure. Uh, one of them is to, to count what, what are the documents, uh, what are the contract types that you're doing. So, uh, transactional document type is an agreement.

20:13Uh, and so what we've done is we've collected all the agreement types. So here is all the agreement types, and they happen to be donated by tiny companies like Thomson Reuters. Noah from, uh, Kira Zuva has donated all of his document types, a- as has not NetDocuments, as has, I think, uh, Radiant Law and ContractPod AI and a few others have all donated all of their agreement types. So what we've done is this is a normalization of all of them put together. So this is a normalized view of all of them, Thomson Reuters and Kira Zuva and Litera and, and, and. So this is all of your agreement types. Uh, and then also what you wanna do is you want to be able to collect all of the c- clause types, which we have here. So a contractual clause, uh, is a thing like, uh, we have about 800 clause types, uh, that are right here, and we're currently normalizing those. Uh, and so we're, uh, debating right now, we have a contract working group that is debating is an act of God clause the same as or different than a force majeure clause.

21:09Maybe they're the same, maybe they're different, and we're working through those things right now. So as a contracting person, uh, stage one is to collect all the document types. That is all the agreement types that you have here. Uh, and then tag up all of your clause types. Uh, and then a third is to be able to say, what are the industries that are involved? Uh, say you're in the information industry, uh, you know, either, you know, telecommunications or, uh, one of these, and you're doing a deal with the agriculture industry. Uh, those are two tags. So then you can say, "What's market for this kind of industry doing a deal with that kind of industry?" Uh, and then what's market for this kind of document versus that document? Uh, and then what's, what's, uh, you know, what's the market for this kind of force majeure clause for this industry versus that kind of force majeure clause? So really, uh, once you tag up the data, then you can get much better, uh, analyses. Uh, and you can go away from anecdata, which is, uh, you know, "I think I'm right," and you say, "You're right," into a more, uh, data-driven approach. So if you are then able to say, um, "I wanna be able to find all the merger agreements that are M&A agreements between banks in the US," and now I've tagged up all of the things that matter in my documents.

22:18And then you could be able to say, "Here's all of the force majeure clauses in all the merger agreements for banks." And then you can run that through a large language model or through a data analytics tool, and then you can truly say, "What is market?" Yes. Um, and, and so this is a data-driven what's market, rather than an anecdata, "I think this is market," versus, "You think this is market." Um, so this is how, again, large language models and tags fit together to do something better than either of them separately cannot do. Yeah, and that, that's very useful because we've seen... I mean, when I was working at the law firm, and when you were through the negotiation process with your counterparties, there's a lot of arguments that it's like, "Yeah, those are market. That's the market clause." And it's like, what the hell do- does that mean? It's like, there's no way you can compare that. But once you have all that information... Uh, we got a really good friend, Otto Hanson, with, with, with TermScout. Otto's great. And they're-- So they're doing-- I love what they did.

23:12They started with, uh, with software companies contracts, so they were doing this type of benchmarking of contracts, and they can do, like, some kind of certification if a contract is, like, you know, balanced between both sides. And that's good because, you know, a lot of the main pain points on contracts is, of course, that negotiation process take forever. And one of the things, like, maybe you can have, you know, the best automation tools, you got the best implementation team for your CLM or whatever, but if your templates are just one-sided, of course you're gonna take forever for negotiate that because nobody's gonna agree or sign a contract with unlimited liability for a contract. They're not gonna-- They're never gonna do that. So it's better to start with something that is more balanced, which is, a lot of the times this is something that, you know, companies do when they hire external counsels to draft their templates. Because of course, external counsels are going to, you know, minimize the risk for their customers and put in all the, you know, the...

24:15Not the risk, but the, you know, responsibilities on the, on the negotiation on the other side. And then if they agree, that's perfect. If they don't, it's just going to take forever, the negotiation process. So that, I really like that part. Yeah. Uh, that's exactly right. And so yeah, so Otto's doing brilliant things. He's, he's a really, really smart guy do- building really big, cool tools. And another person in the contract space is really a smart person d- doing, uh, smart tools is, uh, is, uh, Horace Wu, uh, from Synthia. Uh, he's also... And what Horace is doing, and I think what Otto is doing too, but is doing something really smart, is to be able to say, "Let's put these clauses into vector space." Because even if the clause is named force majeure, maybe within the language of the co- of that clause is a change of control agreement, that, that somebody slipped that into the force majeure clause section. So, uh, the names alone don't tell the whole story, but the language and in vector space, the embeddings tell the full story.

25:07And so I would say that, yes, uh, between tagging up what it's called force majeure, and then pulling the vector embedding saying, "Even though it says force majeure, it's actually change of control," um- Yes... this is probably the way to be able to get to the, the truth of the contract. Yeah, and there's one question from the audience from Raul Magana. This, this a good one that I, that we have to face a lot, especially, like, if we are, like, Mexican lawyers doing, you know, working with US-based clients or UK or, or whatever, and it's, like, the misunderstanding of terms, like translations, and sometimes you use a different term that in the US because of common law or civil law countries they're, they're just different. And so Raul question is that, "Can... Does AI can help to reduce misinterpretation between different legal systems, say common law and Roman law?" I mean- Yes... that's very specific. Uh, uh, absolutely. Uh, and so I would say yes, uh, and, and, uh, partially, uh, because we- we're doing it on the symbolic AI side with Sally, and partially because we can do that in the neural net large language model side.

26:13And so I'll show, uh, each of those, uh, in an actual real example. So on the symbolic AI side, uh, areas of law on Sally, of course, uh, I, we included all of the common law because of course we, uh, I'm a US lawyer and we cl- collected the common law. But then, um, my day job is with VLex. Uh, VLex started in Barcelona, Spain, a civil law country, uh, and they've expanded to, uh, Latin America and North America and Africa and Asia, the Pacific, et cetera. So, um, on the civil law side then, uh, we at VLex had a thing called contract law, uh, and civil contract law, and the property rights and transactions law. These are all areas of law, uh, that are not really common law areas of law, but they are civil law, Roman law, uh, areas of law. And so what we've done, um, what I've been doing is taking all of VLex's thousands of taxonomical items for civil Roman law countries and be able to then put them into the Sally ontology to be able to incorporate those. So for example, uh, you know, uh, there's a, a concept that, uh, we had that is canon law, that is for Roman Catholics, uh, have something that is canon law.

27:19Uh, and so we have, uh, we have now canon law as part of the items here. So this is, uh, something that I'm taking all of the VLex, uh, ontologies and bringing them into the Sally ontologies to be able to better address, uh, what Raul Magana, yeah, yeah, yeah, uh, what he, what is he- ... he's talking about. So yes, uh, but then, uh, and then once you, uh, do that, saying, "Here are all the Sally tags", then you could be able to do the large language model to be able to say, "This is how it is expressed, say, in Spanish. Um, and this is, uh, in vector embedding the idea of that concept is the same as the embedding for the English representation of that idea." And so, uh, so then, uh, you know, between tagging in Sally and the vector embedding extraction of ideas, not words, um, you can get to a better idea of, uh, you know, this is how civil law relates to common law. Great. Pepe, any other, any other questions from the, uh, from the audience? Not for the audience, but yeah. So guys, if you have any other questions, fe- please feel free just to put it on the comments.

28:19On the meantime, I'm gonna keep asking my questions that I have on my script. And I wanna get a little bit controversial now, because we are halfway on our webinar, and, you know, 2023 was harsh for the tech industry. There were over... I saw, uh, I, I think it was on Bloomberg or CNBC, there were, like, over 200 and almost 50,000 layoffs in the industry. And AI was not to blame, but you know, what happened is just, like, companies learn how to become more efficient, and you don't need to use people that takes longer to do certain kind of work, right? Now, in the legal industry, we've seen that even for contracts, I would say even here in Latin America, uh, lawyers are very, you know, agnostic on the technology in to a certain point that they still think that it's more, not that it's more legal, but it's safer to use paper than signing documents using e-signatures. I mean, that's how bad it can take. But we gotta just break that bubble, and we gotta start looking at these tools not as something that to be afraid of, but something that can help lawyers to become more efficient.

29:40And I don't wanna, you know, scare people, but I think there's maybe this year or then on the, on, on the next year, I'm foreseeing that there's gonna be a lot of layoffs. And so you need to be a- ahead of the curve on how to use this, this type of tools on how you can b- become more efficient. And one of the things that I think people are afraid, or maybe they just still don't believe they're, these type of tools can replace a lot of the legal work made by professional with over 20 years of experience working as a lawyer, and one of the main things that they say is like, "Yeah, I use ChatGPT and ask it a couple of legal questions, and it just start, uh, to give me, like, false answers." And there's a lot of, you know, uh, media covering around these cases that, uh, you know, there was, uh, this guy, uh, I think it was in New York, that they, he got some kind of fine or whatever because he was citing some non-existent cases, right?

30:39So I really want to have your experience on some real case scenarios on how, first of all, how can legal teams use this in their benefit? And on the other side, how can, let's say, in-house teams can use this type of technologies to, you know, maybe not leveraging that much into external counsels to do things, you know? Uh, there's a lot, of course, in the market that, uh, companies, you know, legal teams are getting smaller budgets, you know, and, you know, and, and they hate the hourly billing, you know? So law firms need to evolve in how they can offer new types of revenues or, or, or how they can charge their customers. I think that's a whole thing. That, that's a whole world.

31:36We are, we're involving spend management, we're involving AI, we're involving a lot of stuff here. So in a nutshell, Damien, like how can... And let's start with the in-house teams. Like, how can they, you know, become more efficient to use these tools? Uh, so I, I think there are, um, large language models have now really helped to be able to say, "To what extent do I do the things myself? I'm an in-house counsel," and maybe I, uh, the build versus buy is always the question. Do I just build it myself? Do I do the legal work myself, or do I farm it out to my external law firm who can do, do things faster, better, and cheaper 'cause I'm too overwhelmed? Yeah. Uh, that's been the analysis for the last 30, 40 years. Uh, and I think we need to rethink that analysis right now because really, um, why do I farm it out to my law firm? Um, uh, because I'm too busy, uh, because, uh, you know, it might take a 10-hour task, uh, where I just don't have 10 hours to do that work.

32:28But with large language model, that 10-hour task can be maybe reduced to two minutes. Uh, and so maybe it might take longer to send an email to the external counsel than to just do it myself. Um, and so I'm gonna give actual, actual, uh, representations of that, uh, rather than just, uh, abstract, uh, representations of that. So with, uh, with this, uh, so if you have a legal question, uh, what's the trade secret law regarding former employees stealing customer lists? I might have that as a businessperson internally. I might have that legal question. Uh, option one is to call a law firm. Option two is to ask that across actual non-hallucinated cases- Yes... actual non-hallucinated statutes and regulations, and actual non-hallucinated secondary materials, and be able to pull actual non-halluci- uh, non-hallucinated language from actual non-hallucinated cases. So here is actual non-hallucinated language from every single one of these cases.

33:19And now- Damien, can you, just like small parentheses for people that are not familiar with the term hallucination, can you just give like a very brief, uh- Absolutely. Yeah. So that's, that's what Steve Schwartz, uh, that's the problem that Steve Schwartz had. You mentioned the New York lawyer that got in trouble, uh- Mm-hmm... for doing things. And so what he did is he went to ChatGPT, and he asked a legal question like I'm going to ask right now, and asking a legal question of ChatGPT is a bad idea. Uh, the reason it's a bad idea is because, number one, it could actually make up answers. Uh, that's called hallucinations. And, uh, another way to call it is confabulations, which is actually the more proper term. Uh, but what it's doing is, um, when it hallucinates or confabulates, think about if I were to ask, uh, say, Pepe or Marc, ask you, uh, you know, "When you went to college, you read that book. Describe that book to me today in 2024." You might have a vague recollection of it, but you're probably gonna get some details wrong. Yeah. So in the same way, um, me asking this legal question of ChatGPT, it has a, you know, it's probably gonna get most of it right, but it might get some details wrong, factually inaccurate.

34:20So if it, the problem is if you want to be able to ask a legal question of GPT and you ha- if you have to do the legal research anyway 'cause it might get some facts wrong, what's the point of even asking ChatGPT? But with us, uh, you know, asking actual non-hallucinated cases, that is, this case exists in the world. In data science, we call that ground truth. This language exists in that case. That is ground truth. So if you use ground truth to say, "Review the 20 pages of Neville," and here you can see actually the 20 pages of Neville that are right here, um, you could say to the large language model, "From this, how well or how poorly does this text here answer the user's question?" The rate of hallucination or confabulation is next to zero. So that's like me saying to Pepe, not, "20 years ago, tell me about what you'd learned in college," but, "Read this... Hey, Pepe, read this 20 pages right now, and after you read the 20 pages, summarize how well it answers the question." And Pepe is gonna do that really well.

35:17Uh, in the same way, the large language model does that really well without hallucinations. And then you can take all 47 of these cases and turn it into a memo, like you see right here. And when I was a litigator, I would tell my associates, "Don't wait, make me wait to the bottom to give me the answer. Give me the answer right up top." And so here is the answer in one paragraph right up top. Uh, and then I say, "Now that you've given me the answer, dig deeper into the statutes." And here's digging deeper into the statutes. Dig deeper into the cases, and dig, uh, this is where you dig deeper into the cases. And then we say large language models are sycophants. That means they tell you what you want to hear. And so we counteract that sycophantism by saying, "Don't tell me just what I want to hear. Tell me what I need to hear. Give me the exceptions to the rules. Give me the flies in my ointment." So you could be able to say that a former employee can announce their change of employment, uh, and even if their clients are on a protected tra-trade secret list. That is a fly in my ointment that counteracts the large language model's sycophantism, uh, in a way that nobody else does, and we, it does that because we prompt it to do that.

36:16So if you're an in-house counsel and you have this question, option number one is to ask your external counsel to do it. Option number two is to ask your friendly VLex Vincent AI to do it, and this memo takes about two minutes. And so really the question is, how long would it take you to draft an email to your external counsel, uh, and then agree to terms and then to get this answer? So really this is, uh, this is the balance of legal research questions. It's also the balance of legal work. So here is The New York Times, uh, which sued Microsoft and OpenAI over ChatGPT. Uh, here's the complaint from The New York Times. So say you're an in-house counsel that just received this complaint. Once you upload the complaint, we give you these things right out of the box. Because I, as a litigator, when I litigated, I said, "What are the claims that are being alleged? Is it breach of contract? Is it copyright infringement? What are the facts being alleged? What are the roles of all the parties involved?" These are all extractive items. But then we go beyond extraction and say, "Now, I, as a defense lawyer, what can I defend against? What are legal defenses to each claim?

37:19What are questions I can ask my business people to be able to help defend this claim? What are the allegations?" Uh, et cetera. And then we say, "Find similar cases from eight hundred million dockets and documents that are cases like mine, that is copyright infringement cases like mine." And then we have legal research questions that are customized for this particular, uh, document. What is fair use and transformative use in copyright law, especially for artificial intelligence? And so for each one of these, extract all the claims. Copyright infringement is a claim. Here's the statute, the governing law, and the facts being alleged, and here's all the parties that are for this. We do that for every single claim. Then give me the facts that are being alleged. Here's all the facts that are being alleged. Um, who are all the parties involved? Here's all the parties and their relationship to each other. OpenAI is the parent entity, OpenAI LP is a subsidiary, et cetera. Then what is a list of what are they looking for? What is, uh, New York Times looking for? They're looking for money. Uh, they're looking for an injunction to make them stop. They're looking to destroy ChatGPT. These are all things that are extractive.

38:23Then we go beyond extractive and say, "Now, I as a defense lawyer, how can I defend this claim?" Well, for copyright infringement, Vincent says, "Maybe you say that's fair use. Say that OpenAI's use of the New York Times content was transformative. It was for educational research purposes. The amount of the content was minimal and reasonable." We do that, uh, defense for every single claim in this matter. Then we say, "What questions should I ask my client, OpenAI? Hey, what datasets did you use to train the models? Um, did The New York Times copyrighted work? Were they in there? Um, what process, uh, did you select and compile those, uh, datasets? Did you seek licenses be-before you did this? What did you do to prevent inclusion of copyrighted material, if anything?" These are the right questions to ask, and it spits that out for every single claim in seconds. Here's a legal research question. We can give you that same kind of memo. Here is, uh, I'm a copyright lawyer, so the idea-expression dichotomy is an argument you can make, saying that the foundational model is extracting un-copyrightable ideas. Uh, those are called vector embeddings.

39:24And copyrightable expressions are thrown away. And then here is now my motion to dismiss, arguing exactly that, that here's how vector embeddings work. Uh, here's how ideas, uh, are not-- are the thing, not expressions. We throw away any copyrightable things. Then we say that The New York Times, when they provided the first eight paragraphs of The New York Times article, actually performed a hacking exercise. Uh, and that's where bad actors try to get the system to do something the system shouldn't do. Uh, and in copyright actors, the bad actors are the primary infringers, and the tools, in this case, ChatGPT, are at most the contributory infringer. And for contributory infringement, under the VCR case, the Sony Betamax case, is whether the tool, in that case, the VCR, in this case, GPT, is it capable of substantial non-infringing uses. Um, and can your VCR can be used to record your kid's baseball game? Can ChatGPT be used to be able to create poetry, right?

40:16These are all good arguments that now here is a good argument saying, "Your Honor, this is what The New York Times did. They hacked. Uh, they acted as a bad actor, and this is the Sony Betamax case, and this is how I win." Um, find similar cases in Docket Alert. Here are five similar cases of, uh, against OpenAI and other, uh, op- other LLM-based copyright cases, uh, that are just like mine, where I can take the answer from this. I don't have to be a prompt engineer to do any of these things. Look, I just uploaded this, and it gives me all these things out of the box. So if you're an in-house counsel, option number one is to send this to your outside counsel. Option number two is to run it through this. And as we talked to in-house counsel, they said, "You know what I'm gonna do?" And this is like Fortune fifty in-house counsel. They said, "What, what we're gonna do is we're gonna take every complaint that we've been s- filed, uh, over the last year, and we're gonna run it through this. And then we're gonna compare the output of this to what my lawyers did on top of this. Did my lawyers do anything on top of this?

41:14And the, the answer is no, they didn't do anything on top of it, that's gonna say something. And then going forward, uh, what we're gonna do," they said, "is to be able to, before we even send the complaint to our law firm, we're gonna run it through this. And then we're gonna give the output of this to my law firm and say, 'What can you do on top of this?' Because that's all we're gonna pay you for." And this is not just a, uh, litigation thing, but it's also a, uh, transactional thing. Here's a merger agreement where LinkedIn, uh, uh, got acquired by Microsoft. So I wanna identify inconsistencies, ambiguities in overlapping con- uh, contracts. Um, give me the definitions, uh, to extract them and verify that they're used properly throughout. Create a condition-- a checklist of conditions precedent, things I need to do before closing that it must be satisfied. What are the representations and warranties made by LinkedIn in the agreement? Um, give me the covenants and obligations imposed on LinkedIn. These are all customized for this. I don't have to be a prompt engineer. And then here are legal research questions.

42:12What are the legal consequences if a party to a merger breaches a confidentiality, uh, under Delaware law? Which is what this is under. Uh, under what circumstances? And for this, you need, uh, to do these legal research questions. You need the statutes. You need the regulations. And, uh, you know, to do that, uh, you could imagine, uh, as I go into here, if you don't have statutes in all the jurisdictions, you're not gonna be able to answer these questions. So for example, you could be able to ask this question about establishing and operating franchises, um, across California law and New York law and United Kingdom law- Spain... and Spanish law and European law. And if you want, you can be able to do that in-- add other jurisdictions, like maybe- Mexico... uh, Chile, Mexico, uh, and then, uh, other jurisdictions here. So now you could be able to get not just one memorandum, but you can get many memoranda. One for California, one for New York, one for the United Kingdom, one for Spain.

43:10And then you could be able to say, "Here's now this question, and I'm asking, I'm translating this English question into Spanish." And now I do that vector embedding search in Spanish, and then I round trip it back into English. And it's done that because I, uh, set my language settings to English. I could have just as well said I'm a native Spanish speaker, um, so I can ask a Spanish question across French law and then get a Spanish answer to my Spanish question. This is, uh, s- we're the only game in town that allows that, and we're the only o- game in town that allows this multi-jurisdictional query to be able to ask this question. And you might say, "This-- Damian, this seems like a 50-state survey." Uh, and I would say it is, but it's not just a 50-state survey, it is a 50-country survey. And you need to do that 50-country survey, you need the data, the oil of the 50 countries. And if you look at the United States, um, only three companies in the world have comprehensive cases, statutes, and regulations. That's Thomson Reuters, my former employer, LexisNexis, and us. But then if you look to Latin America, how comprehensive is, uh, the duopolies in, uh, Ecuador, Brazil, Argentina, Mexico, Peru, uh, Costa Rica, Colombia?

44:21Uh, how extensive are the datasets in Spain, Italy, France, Austria? How extensive are they in the tax havens of Jamaica, Bahamas, Cayman Islands? How extensive are their collections in Hong Kong, the banking powerhouse, Malaysia, Singapore? Uh, so if you think to the United States, only three companies in the world have that data, uh, Thomson Reuters, Lexis, and us. But if you go worldwide, there's only one game in town with the depth and breadth of vLex. Uh, and you might have noticed that I didn't mention, uh, Casetext, uh, CoCounsel, uh, and you might say, "Where did Casetext get all of their data?" And you may or may not know that they got all of their data from us. That is every day we would give Casetext our cases and statutes and regulations. Uh, and so when you really want to trust it to be able to answer these questions here, or if you want to trust it to be able to answer, say, contract questions, what are the legal consequences if you breach your confidentiality obligations? If you don't have Delaware law, that is you don't know the case that just came down yesterday, uh, for this, and you're relying on Harvey or somebody else, uh, that doesn't have that, Harvey's gonna give you a bad answer, 'cause they don't know about the case that just came down yesterday.

45:31Um, so really the oil is the thing. The research, the cases are the thing. Uh, and so this is, uh, this is something that I think people don't realize as much, uh, that data is the new oil. Uh, we've known that for a bunch of years. And if you, um, if you want to be able to, uh, you need to extract that oil, uh, and you need to be able to, say, refine it into, say, gasoline, pipe it to customers, figure out what the product is, and get it to market. So legal data is, uh, statutes, regulations. Uh, it's court data. Uh, what are the court decisions that just came down yesterday? Uh, these are all p- public oil. And then of course privately, within your corporation or within your law firm, you have contracts that are private. You have settlement agreements that are private. These are private oil. Uh, you take these data, and then you have to summarize them and tag them up. It used to take thousands of people to do that, but now with large language models, you don't need thousands of people. You just use your friendly Sally tags and then tag up and summarize those things. Then you take that tag data and do you make a website or do you make a data, uh, str- stream?

46:29That's-- both those things are easier than ever with large language models. And then you have to decide what are you gonna build? That is, uh, you need smart lawyers to say, "What are the pain points that people will pay for?" Uh, and then you say, "I need a good product and I need a good user experience." It has to be Apple-like in simplicity. There's some value there. And then of course it's always about the relationships. Do I trust that person? So if you think about moats, that is a competitive advantage, it costs about two billion dollars to download the case that just happened yesterday, to download PACER, uh, et cetera. That is a big moat, two billion dollars. But then the moat for, uh, tagging up and summarizing, with large language models the moat is zero. There's no moat there. So if you are to hiring a vendor that is merely summarizing and tagging, what's the point? Because they don't have your private oil. They're running it across your private oil, and they don't have the public oil either. They don't have the case that just happened yesterday. So what's the point? There's no moat there. There's no moat to creating a website or a data feed.

47:26There's a minor moat, I guess, kind of a, a bigger moat to say, "What do I need to build?" That is, what is the pain point I'm doing? There aren't a lot of lawyers and tech people that are good product people, that are good UX people. There's a moat there, and of course it's always about the relationships. Uh, you want to be able to trust the people. Are the people charlatans that are just, you know, trying to go on this AI craze, or do they actually know what they're talking about? Do you trust them both what they're doing today and what they'll be doing t- tomorrow? Is this relationship worth it? So if you are thinking about hiring or you have hired somebody that is merely a distillery, a refinery of oil, think about what's the point, and maybe you could just distill or refine it yourself. Anyway, that's a, that's a long, uh, long discussion, uh, to your very short question. And, and I mean, Damian, I was gonna ask, you know, to, to wrap things up, you know, if there's people out there that are curious about this and they're like, "Hey, well you know what? We, we can use, we can leverage something like this," right? How, how do they go about getting, um, buy-in from other parts of, of the organization?

48:23But I, I think you just showed a lot of that on that, on that last r- slide, right? That's the ROI. Uh, and also talking about, you know, for those in-house members out there, literally sending out those, those documents that say, "Hey, we, we already have all this information" to, you know, to their outside counsel. Can you go above and beyond this or, or is this all you're gonna give us as well? Any- anything else to add to that, you know, for, for organizations that are out there, w- what they're looking at, at Sally or they're looking at, uh, other pieces of technology to add to their, to their tech stack a- a- about trying to get buy-in from other parts of the organization and funding? Yeah. I, I think that, uh, the industry is moving so quickly right now. GPT-4, uh, uh, GPT 3 and a half in December of '22 beat only about, uh, ten percent of humans on the bar exam. GPT-4 beat ninety percent of humans on the bar exam just three months later. Uh, so really, uh, that rapid, uh, expansion I would wager is going to be the case with GPT-5 when it comes out maybe later this year. And so, um, so Think not only about what your company should be doing today, but think about where the industry is going, and think about how many of the vendors that are selling you things are solving problems that are maybe GPT-3.5 problems or GPT-4 problems that will be solved with GPT-5.

49:34Uh, one of those is context window. GPT-3.5, you ha- you could only do little chunks of a document, and so things like LangChain came out to say, "Okay, now I can chain together those little chunks." But now, uh, then Claude came out with a 200,000 token limit, which you can put a whole book into it. So that solved that problem. Now, Gemini just came out two weeks ago and said- Mm-hmm... "Uh, n- we're not doing 200,000 tokens, we're doing a million tokens." Uh, so that can be able to do many War and Pieces, right? Uh, and so really if you tried to solve for GPT-4, all of a sudden that's a moot point. Uh, so you have to think what is your vendor solving for, and is that going to be changed next week or next month? And so really the thing that can't be solved next week or next month is the oil. You're not gonna be able to download yesterday's case. You're not gonna be able to download the regulation that just came down yesterday in Spain and in Bolivia and in, uh, Peru. All right? That is something that is continuing. Uh, even as the technology gets better, the oil is the thing. Mm-hmm. Uh, so really think about who has the oil and who is merely refining the oil.

50:36And I got two more questions because we only have a couple of minutes left, and this was a question from a good friend of mine. She works in a huge company in the US, and one of the main concerns on why they're not adapting or they're not buying this type of technology, it's because of privacy matters. Like, they don't want their data to be shared with, with anybody. Like, can you answer that concern? Yes. Uh, so there's, uh, there are many people along the chain that you have to trust to ensure that your privacy is maintained. Uh, we at V- vLex, we tell you that we will never, ever train on your data. Uh, we can give you that commitment from us. Uh, we also require from every single knowledge gra- knowledge large language model that we work with, that they will never train on the data of our users. Uh, so we work with, uh, you know, GPT-4 obviously, Claude 3, Claude 2, uh, Mixtral, and a whole bunch of others. Uh, and for all of those, uh, we say that we will never, ever train on the data. So you have to trust us at vLex. You also have to trust the endpoints, uh, OpenAI, et cetera. Um, and so if you can trust both of those things, then you're probably fine.

51:39But of course, the, the bigger the organization, that is vLex is a big organization with big, uh... Then the more bargaining power you have to say, "Do not train on my thing," and the more they'll listen to us maybe where they might not listen to a smaller player. So in this case, uh, maybe size matters. Yeah. And the last questions, because I think this is going to be, well, it is already a revolution, and I'm thinking about like having Vincent AI when I was a, a student at law school, I would get straight As in all my exams. So I remember a couple of years ago, be- before the COVID, especially like in Codex in, uh, at Stanford, there was a big discussion on whether lawyers should need to learn how to code in order to become more tech-savvy. I think right now you don't need to, you just need to do, like, some promptings. And so in your experience, Damien, Damien, in thinking about maybe even as a student or as an active professional or as a lawyer, how can you learn how to use this type of technology? And, yeah, in the one minute we have left, uh, two answers to that.

52:48Uh, number one is that, uh, um, I've heard from coders, and it's true that the most important programming language today is English, uh, with apologies to our Spanish-speaking audience, that it re- literally to be able to prompt is just speaking English, much like- Yeah... a regular human would. So that's thing number one, you need to speak English. You don't need to code. Thing number two, Sam Altman on stage was asked, "What do you think of the ascendance of prompt engineer as a field?" And Sam Altman said, "Ugh, I hope that's not a thing in a year, because if we're doing our jobs at OpenAI, uh, it, you won't have to be a prompt engineer. You'll just simply ask a question, and we'll give you the answer." So in the same way, I at vLex have the same thing. Um, you don't prompt engineer in vLex. All you do is you ask a legal question, much like you would ask it in an email to an associate, much like you would ask it in an email to your associ- uh, to your external counsel. So you don't really need to prompt engineer. Um, all you need to do is ask this question.

53:38So anyway, the tool makers, I need to prompt engineer, because what you don't see behind the scenes are the 25 really smart prompts that I do on my side, but the end users don't have to. Um, so I would say unless you want to work for a legal tech company like I am, you don't need to prompt engineer. You just need to be able to ask questions, and if we're doing our jobs right, you don't have to. Awesome. Well, I, I think, you know, as we have seen, the future is here, but it, it's exciting to see even further into the future and, and, and what that next stage of, of growth is going to, uh, going to have for us. But, uh, Damien, this has been a fantastic conversation. Really appreciate you coming on and, and walking, uh, those folks that stayed on to through the application and, uh, and through some different scenarios. You know, thanks again. If, if folks wanna connect with you or, or learn more, uh, where's the best place to do that? Sure. Uh, so, uh, yeah, to, to the future is here, uh, that there's a saying that the future is here, it's just not evenly distributed, and I think that's where we are now.

54:30And as far as the how to connect with me, LinkedIn is probably the best spot. Uh, I also hang out in X and Twitter every once in a while, but LinkedIn is the primary. Awesome. Awesome. Well, thanks everybody for joining us, and, uh, thanks for listening to another, uh, Contract Heroes webinar. Thank you guys. Have a good one. This episode of the Contract Heroes is sponsored by Koho Consulting. Koho helps organizations of all shapes and sizes find, implement, and manage their chosen CLM tool. Whether your project is buy or sell side led, Koho can help you navigate the waters of the CLM space to make sure that you're getting a product that's going to fit like a glove for years to come. If you'd like to learn more about Koho, visit their website at C-O-H-O consulting.com.