4 The Record Podcast

Noah Waisberg on the Future of Kira Systems and ‘AI for Lawyers’

Also Available On

Episode 8 Transcript

Erin Harrison:

Welcome to 4 the Record, where we go on the record with today’s leading influencers and experts to discuss the latest trends in legal innovation and the business of law. I’m your host, Erin Harrison and today we’re joined by Noah Waisberg, the co-founder and CEO of Kira Systems, the AI-powered contract analysis company based in Toronto. Noah, welcome to 4 the Record.

Noah Waisberg:

Hey Erin. Thanks so much for having me. I’m really glad to be here with you.

Erin Harrison:

And hopefully I have that right. I know your roots are in Toronto, but you’re all over.

Noah Waisberg:

Yeah. That is right. I’m from Toronto and the vast bulk of our people are here, but we have people in other places – a bunch in Europe and a bunch in the U.S. too.

Erin Harrison:

So, first of all, I wanted to say congratulations on Kira’s 10th anniversary and also on your new book, AI For Lawyers, which we’re going to talk about in just a little bit.

Noah Waisberg:

Thank you. Both took a lot of work.

Erin Harrison:

Undoubtedly. I think there’s been a lot of hype around artificial intelligence and AI. I think we’re seeing some of that hype turn a bit more into a reality as we’re seeing some of the more pragmatic ways AI can be applied in legal, a lot of which we’re seeing in task-based or repetitive work. So with that, my first question: how did you and your Kira co-founder, Dr. Alexander Hudek, decide that now is the right time for your book?

Noah Waisberg:

Well, what is the right time or what isn’t the right time? What we saw was that, first, lawyers are using AI heavily. It’s not just hype, but there are areas of law practice where AI is in pretty regular use. We thought that was an important story to tell, that this is something that’s here, like it or not, and you really don’t have a choice about whether AI is going to be part of law practice or not. That’s not the case across every area where people are talking about using AI. Certainly there are areas of hype that aren’t yet matched by reality but the overarching story is that there is a lot of AI usage in law already, and we thought that was one important message to get across.

And then the other one is that this doesn’t have to be a bad thing. This can actually be a really positive thing for lawyers if they choose to embrace it. So, that was the message that we were to get across. Is it the right thing to put this book out in 2021, as opposed to 2020 or 2023, I don’t know. As a practical matter, it worked out fairly well for me because I do not know how I would’ve gotten the writing. The book was planned well before COVID, but I typically travel a lot, one of the pluses or minuses of running a business out of Toronto is that probably about 95% of our revenue is non-Canadian and our customers like to see me a bunch, so I end up traveling a lot. So I practically do not understand how I would have done the ton of work that it took to do the book if I’d been traveling on my regular schedule.

So it worked out well, having written it over a lot of 2020. Definitely watched a lot less episodes of Succession than my wife and I would have liked. My summer was less relaxing and all that, but I think this is a good moment to have it out and we’ve been seeing that lawyers seem to be pretty receptive to the book. So hopefully it was the right moment, but it was our moment. I don’t think that much thought came into it has to come out…

Erin Harrison:

The timing.

Noah Waisberg:

Right now, but it did feel right that there was enough of a real story of what was happening in AI that we could start to do it and have lawyers engage with that.

Erin Harrison:

Yeah. An inflection point for the industry too, I think, because everyone was trying to figure out how they were going to just function on a day-to-day basis. So it seems like it’s coming at a good time now where everyone’s now looking forward, but had that little bit of a pause in between the hype cycle.

Noah Waisberg:

I think that’s right. Actually, it’s too early to tell. Here we are recording this in March, so I don’t know yet how 2021 is going to play out. But one of the things that I have heard from some smart people and gotten a sense of just from the market is that in 2020 a lot of organizations had a lot of technology base change to deal with and they did it. And now my sense is there are places that are starting to think about well, what else can we do differently? And so hopefully this is a really good time to have it out, but it’s not like we get to run the simulation multiple times and see if only we’d put it out in September 2020 when everybody was at home, it would have done better. I don’t know, but it seems like we’re getting positive feedback on it so far and people are reading it, so I’ll take it.

Erin Harrison:

So I’ve looked at the book. I’ve read through a fair amount. I see that it has both anecdotal information on how law firms are using AI and also a practical how to framework for actually implementing the technology. Could you expand a little bit about how you approached that?

Noah Waisberg:

Yeah. So first off, thanks for reading the book. Hopefully you enjoyed it. We really tried to make it as close to a page-turner as you could make a book about AI for lawyers. So hopefully it was a pleasant however many hours you spent with it. In terms of what we’re trying to say with the book, there’s really two messages, as I was saying before. Number one, AI is here in law, like it or not, it just is. Number two, here’s what that means for your lawyers and that can be positive. What we tried to do in terms of the structure of the book was, first off, just talk about the theory, objections that we’ve heard over a decade of selling this software that lawyers have to using AI and their responses.

The economic case for how doing work more efficiently, even if you are an hourly billing lawyer, can actually work in your best interests. How this plays out, especially not just for large firm lawyers, which are where we come from, but we brought in a couple experts, Sam Glover, Carolyn Elefant, to help us approach how AI might be thought of by smaller firm lawyers, so we could have greater empathy for that group of people, how this is going to impact training and even the distribution of lawyer knowledge and ethical issues. That’s the first part of the book.

Second part of the book is some case studies into a few specific areas like e-discovery contract analysis, legal research, expert systems, litigation analytics, where there’s heavier use today of legal AI, so that just people could get a sense of what’s possible and what’s going on in the most adopted areas in the area.

The third part of the book is more about thinking about adoption and where you’re at with legal tech adoption and legal AI adoption. One of the things with the book is we really, we phrase it in the introduction, that we’re writing more a book like A Year in Provence than we are the [inaudible 00:08:13]. We’re not telling you this is the AI that you should buy or here are the specific questions you should ask if you’re buying a contract analysis AI, but rather tried to get people intellectually engaged with the idea that AI is here and that it can help or hurt them.

Erin Harrison:

One of the highlights of the book is looking at why AI is now so vital in the legal workspace. Would love to hear why that is and more specifically, what can AI do for contract analysis technology?

Noah Waisberg:

I think why AI is so essential is I don’t think you can do a good job as a lawyer in certain areas if you’re not using AI. So the area that I know best is mergers and acquisitions. I used to be a mergers and acquisitions lawyer at Weil Gotshal in New York prior to co-founding Kira. So if you’re doing a typical M&A deal, say $400 million deal, you’re probably going to review somewhere from 75 to 500 contracts as part of the diligence. You’ll review the so-called material contracts; because in fact, that company probably has 10,000 contracts. Material just means the important contracts. You’re going to review the important contracts out of the 10,000. Out of the 10,000, you’re going to review 100. That’s super dangerous. There can be important stuff in those other 9,900 contracts that you’re not reviewing and you would never find it.

I think a lot of corporate lawyers would say, “Blow a change of control or an assignment clause, no big deal,” but you could be missing an exclusivity clause or most favored nation pricing term or a non-compete or an unlimited limitation of liability or some wild indemnity. And you would never know it was there and your client could end up bound by this and really in trouble. And we’ve heard of this happening and it does happen. And it’s happening because lawyers aren’t able to access this whole scope of the documents because no one feels like spending the time it would take to review 10,000 contracts. But in fact, if you use the technology, you can do that.

Same story is playing out in legal research. You’re not going to be able to find relevant cases if you’re not assisted by AI. Same thing in e-discovery. If there’s millions of documents, you’re probably not going to turn over the right ones or spot the right ones if you’re not assisted by a computer, unless you have a $50 million budget and a ton of time. AI is just enabling lawyers to do more of the work that they should be doing, to practice law the right way, unconstrained as much by time in the way it is without AI.

Erin Harrison:

I read a couple of the case studies referenced in the book, one of which was from Meredith Williams-Range, who’s the CKO, chief knowledge and client value officer, at Shearman & Sterling and she said that after using Kira for one year, her firm found that when humans and machines hold hands, the rate of accuracy was better than either one independently. Could you talk a little bit about the risk of human error and how AI actually helps mitigate that?

Noah Waisberg:

For sure. Beyond the fact that humans just aren’t looking at everything – I think that’s the really big error that a lot of people aren’t thinking about right now. Even in the status quo process, it’s riddled with potential for error. And I think if you speak to someone like me, who’s been a mid-level associate or a senior associate or junior partner at a good big law firm, they’ll all have had the same experience that I did of just it’s Thursday night at 9:00 PM and you’re reviewing the diligence, reports from a bunch of juniors that you need to put into some format and send over to your clients the next morning and realizing that the people who were on the project didn’t realize that a director indirect assignment is a change in control. So that happens all the time.

The way that I like to think about it with respect to due diligence is there’s both systematic and random error components. So the random error component of human error is just that you’re doing the work at 4:00 AM, you were doing the work at 4:00 AM the night before and the night before that, you got in a fight with your girlfriend or boyfriend, you’re distracted, Kim and Kanye have just filed for divorce and it’s big news and on your brain. Any of those things can make the people who are doing this work miss something that’s there. Then there’s a systematic error component. So that’s just random, you don’t know. It’ll be worse when people are a bit more tired, bit more distracted, you don’t know if someone got in a fight, who knows?

Then there’s a systematic component, which is contract review is typically done by very junior lawyers. Good firms – these are junior lawyers who went to the best schools. They did well at those schools. The firms spend all the money they can on training these juniors; but nonetheless, they really don’t know what they’re looking for in a lot of cases. So if you had more senior lawyers doing this work, they would actually probably spot these things; but the junior lawyers who are doing the work, who are super smart and as well-trained as they could be, just don’t know what they’re looking for and so they make mistakes anyway. So there is a level of human mistake. And I think everybody who is involved in the diligence process and honest about it knows that is there.

Noah Waisberg:

But I think there’s a deeper problem, which is the one that I was describing a second ago, which is humans miss a 100% of the things that they don’t look at. And I think that is the norm in today’s contract review. So AI can help you make your standard process better. And we’ve heard that over and over again, and experienced that ourselves of our software finding stuff that a lawyer might miss, even a careful lawyer, even an experienced lawyer. I’ve seen it catch strangely worded things like exclusivity clauses or MFNs that I’ve missed. And I think people who used to work with me would have said that I was a careful lawyer, but going beyond that, you can’t do the work if you’re at the same level of accuracy, if you’re not reviewing everything, and the technology enables you to do that.

Erin Harrison:

Yeah. I work with a number of legal tech organizations who are selling into law firms or selling into corporate legal departments and I think you do hear that a lot, like, “Well, how accurate is the AI?” But as you said how accurate is a human doing the work at four in the morning…

Noah Waisberg:

Sorry to cut you off but I think the way you need to think of it is the same as what you’re going to be using the AI in practice. So if you’re going to be using the AI without a human supervising it then the relevant question is how accurate is the AI versus a human. But for a lot of our customers, they’re using the AI as a supplement to human lawyers. And so the appropriate thing to think about, which Meredith talks about, is how accurate is a human versus how accurate is a human supplemented by the technology?

Thing that we hear from our firms more often is not that the AI is more accurate than humans, but rather that it’s no less accurate. And I think then there’s some people like Meredith who are especially forthright, who will say that they actually see it being more accurate. And that’s certainly been our experience when we’ve run tests.

Erin Harrison:

That same case study with Meredith with Shearman & Sterling also talks about building trust, which was what I was just going to ask about. I think we all know that the legal industry has not been the quickest to adopt new technology and I think there’s still a fair amount of skepticism. What are your thoughts on how we as an industry can overcome some of those psychological burdens that would help improve adoption?

Noah Waisberg:

Selling efficiency technology to lawyers is probably not as easy as selling efficiency technology to car manufacturers or something like that, but I don’t know that I buy that lawyers are as bad with adopting technology as they get a rep for. I think of a whole bunch of pieces of technology that have been adopted pretty ubiquitously across law, and that’s happened in recent years as well.

When I started as a lawyer in 2006, virtual data rooms were getting to be pretty heavily used, but they weren’t used all the time. I remember going to Pittsburgh and St. Louis, and going into a physical data room and pulling out boxes of documents and reading those. When I talked to people who were five years ahead of me, six years ahead of me in practice, they had lots of stories. People who started around 2000 and ’99, at that point, virtual data rooms were becoming a thing, but they weren’t a ubiquitous thing. They were lightly used. By the time I got to be a lawyer in 2006, they were getting to be more heavily used. Obviously in 2021 virtual data room usage is ubiquitous but I think even by 2018, 2019, it was pretty rare, seriously rare, that anyone was going to a physical data room.

So it is with redlining software, blacklining software, DeltaView, whatever you feel like calling that category of software. When I talked to partners that I used to work with, they all had stories about either doing the work themselves or sending the work out to a secretarial pool within the firm of comparing documents. When I was a lawyer in 2006, that didn’t happen. It was just ubiquitous use of blacklining software. I don’t think that transition was immediate, but it was ubiquitous.

I think there’s a lot of other examples like that – electronic legal research. My dad used to be a lawyer and there was a library floor in his firm. I think now there’s still lots of librarians at firms, but they’re in offices, they have computer terminals and maybe there’s a few books somewhere. BlackBerrys, email, tons of examples throughout time of lawyers heavily adopting technology. As someone who builds the technology, I think it should be used ubiquitously right now. I know that lawyers can do a better job doing diligence if they use their tech, but it takes time. The thing that I feel encouraged by is seeing how heavily lawyers have adopted technology once they’ve adopted in the past and I don’t think that’s so in every single area.

If you ever read things like Atul Gawande’s Checklist Manifesto talking about trying to get doctors to adopt new ways of doing things, like a greater use of sanitation techniques. It can be a struggle 50, 60 years after the fact. And that’s in area where there’s significantly better evidence of the benefits, where there’s peer reviewed, heavy scientific studies and it still can take 50 years to get some of these innovations in. So I think lawyers may be better than we give them credit for. Even though in the specific case of contract analysis technology and AI, I’d love it if everybody was using it ubiquitously right now. I think it’s going to get to that spot, but it does take some time.

Erin Harrison:

I actually agree with you, Noah, that lawyers do get a bad rep because I think as this evolution is occurring within legal tech, we’re starting to see lawyers and heads of innovation looking at other use cases for the technology. Another one of the use cases I’m referring to was in regard to Perkins Coie, A Retiring Partner and Knowledge Transfer, and I thought this was a really compelling story because we often think about using AI for contract analytics or legal research, but don’t realize the way we can actually train these machines to learn what an individual person that knowledge has in their head and to be able to apply it elsewhere.

Noah Waisberg:

Yeah. I love that story. The way it is written, it’s a little hard to tell, but in fact, that was actually from Amy who told that story’s previous firm. So it wasn’t a Perkins Coie story but I think it is something that’s really powerful just in how AI can help lawyers amplify their expertise. So right now lawyers, basically, in most cases, only work on the projects that they work on. In order for them to make money, they have to work on a specific project unless maybe a couple of them write books, a couple of them do, maybe, a form that gets distributed more broadly, but the primary way that a lawyer makes money is working on a specific legal project.

Compare that to music recording – So 150 years ago, if I felt like hearing music… Just before we got on this podcast, I was listening to a song. If I felt like doing that 150 years ago, pre the gramophone, I would have had to… I don’t know. My kids are out, one of them can play the piano nicely, maybe I could get him to play me a song or something like that, or I would’ve had to find my wife and get her to sing to me or something, or maybe a wandering minstrel would happen to pass and I could get them in to play. Those were the levels of choices that I had. And what we’ve seen as music has gotten recorded is that there’s been transition in artist careers. They’re always doing different things, but for artists who are at the top of the game, they’ve been able to make way more money because I can listen to, I don’t know, Vybz Kartel or whatever it was, right here from my Spotify and just distribute that really broadly.

What AI enables lawyers to do is to capture their expertise in a system. So in the early days of Kira, I spent about 80% of my time for the first couple of years of the company teaching the software to find new concepts like change of control, assignment, exclusivity. So I would read a contract and I’d point to the exclusivity language and from that the software would learn that. I always took that super seriously because I knew that someday there would be a junior at a place like Davis Polk sitting there at 4:00 AM with me, effectively, on their shoulders saying, “Do you think that’s an exclusivity clause?” It was like bottling up my expertise and putting it there so that a first-year someday would have that available to them. I think more and more lawyers are going to be able to do that, encapsulate part of themselves in AI and be able to actually deliver their expertise even when they’re not there.

If you think about the amazing thing of that example I was giving with the 4:00 AM at Davis Polk, it’s like I was working, but I was asleep in bed. In fact, I could have been working at 20 different places at that very moment. Instead, I was sleeping and people were getting value out of my knowledge. And I think that’s something that’s really remarkable about AI, especially for people who are great lawyers. If you have a real subject matter expertise, it offers the opportunity to share that with others without having to physically be there in person. And I find that super exciting.

Erin Harrison:

And super interesting. So just shifting gears a bit now, Noah, I wanted to talk with you a bit about Kira and the strategic restructuring that was announced earlier this year. How has the focus shifted for Kira and what was behind the restructure?

Noah Waisberg:

Well, I think as a company you need to sit and constantly be thinking about what are the things that we can do better than anyone else in the world and how can that create advantage for us? As a company, we actually started 2020 at about 180 people. Even after the restructure, we’re still close to 200. So things are okay. As we sat and we thought about where were areas where we were doing great, where it was easy to sell more of our software or where we thought we had potential to sell more of our software, and where were areas where we thought they might work better and they didn’t or maybe where they overextended our team. It wasn’t easy. The market wasn’t as ready yet.

An example of that is we found when law firms use Kira, about 90% of their usage is in areas that we’ve taught Kira to find information out of the box, so Kira’s software that reads through contracts and pulls information out automatically. We’ve taught it to find 1,200 data points or so, out of the box in English. So you can put in a contract and it’ll like 1,200 different things or has the potential to find more than 1,200 different things in that contract. Our customers have taught it to find, I think, something like an additional 20,000 data points. But one of the things that we found as we got customers who extend beyond law firms, we’ve got accounting customers, consulting customers, we’ve got corporates themselves who use the software. But one of the things we found when we looked at law firms was that with large law firms, about 90% of their usage of Kira was on stuff that we’ve taught it.

One of the spots where we’re thinking about is in continental Europe, we’ve got a bunch of continental European customers, a bunch of places have been successful, but it’s a harder sale because if you’re a Dutch law firm, we have several Dutch law firms who are our customers, if you’re going to subscribe to Kira, you need to train Kira to find information in Dutch, or you need to use it on English contracts. And so that’s fine if you’re using it on English contract, but it’s a significant commitment to train our software. The training’s pretty easy. We think we have very data efficient algorithms in terms of how much training data they need, but it takes real work to train it in Dutch. And so what the corollary to that is while we’ve been able to bring on some Dutch law firms and it’s been successful, it’s a harder sale.

Because not only do they have to pay for the software and figure out how they’re going to roll it out, but they also have to teach it. And so we found there that it was like pushing a rock uphill up a steeper slope. And so what we’d like to do is just push a rock ideally downhill, but you’re selling tech to lawyers. It’s not always that. We think we have. So we’ve got a couple of products rolling out later this year and one of the things that we really felt like making sure we did is that we had enough bandwidth and resources that we could make sure that those launched okay. And from what we’re seeing with one of them, it actually seems like it is maybe like pushing a rock more downhill.

Noah Waisberg:

People seem excited about this so what we were trying to do, and I think is a good exercise for every company to do on a regular basis is just think about where are the spots where you’re thriving, what are the few things that you can do that’ll make a really big impact and then what are the things that you’ve tried that seemed like a good idea, but you shouldn’t be doing it. So what we decided to do was just do a little bit less stuff, but try to put more resources on those things.

Erin Harrison:

And now looking ahead, even beyond 2021, what’s next for Kira? What’s going to be the main focus for you and for the company 12 months from now?

Noah Waisberg:

So we’ve got two things we’re trying to do in 2021. Number one, we believe that every time a lawyer reviews a contract, they should be using software like ours, ideally ours. And so what we’re trying to do in 2021 is just drive towards ubiquity in law firms, ubiquitous use of our technology in law firms. Anytime they review a contract, they do it using our tech. So that’s the first thing that we’re working on. Second thing that we’re working on is we’ve got a pair of products that we’re trying to get launched this year and get traction on. We’ve been listening to the market and thinking about what makes us really special.

I think we are the largest company most focused on pulling data out of contracts and by a fair degree. The next closest company is not close to us in terms of size and maybe not even in terms of focus. So we think that focus gives us a real advantage and creates a lot of opportunities for us to do additional things. And so we’re trying to get two products out. We’re hearing positive, enthusiastic feedback about especially the one that’s closer to market so far, and we’re just excited to have those out and hopefully we’ll be able to get traction on them. So those are the two things, just ubiquity in law firms and get traction on a pair of product led launches.

Erin Harrison:

That’s great. And before we wrap, Noah, where can people buy your book?

Noah Waisberg:

So here we are in March 2021, you probably buyyour books online. You should be able to find it at your favorite online bookseller. I know we’ve had success across a lot of different national Amazon outlets for getting it, but if Books-A-Million is your thing, go for that too. But it should be available. As we did the book, we tried to make it something that someone could get a lot of value out of with three, four or five hours on a Sunday afternoon. And while I’m no Michael Lewis, we try to make it a pretty fun read. And so do pick it up, get it on Amazon, you can probably have it delivered by the weekend. And hopefully it will be an entertaining read for you then.

Erin Harrison:

Yeah. And again, the name of the book is AI For Lawyers. Thank you, Noah, for speaking with me and thanks for our listeners for joining us for this episode 4 the Record.

Noah Waisberg:

Thanks so much for having me Erin.

Erin Harrison:

Thanks for listening to this episode of 4 the Record. You can listen to more episodes of 4 the Record wherever you listen to your favorite podcast or go to platform.com/4-the-record. Platform helps establish leaders and emerging growth companies articulate how cutting edge technologies and services are reshaping and re-inventing the world we live in. Until next time, this is Erin Harrison.