How AI is powering legal research at CanLII

CanLII is harnessing the power of artificial intelligence to help simplify and optimize legal research with its new AI research tool, Search+. The tool is designed to convert plain language questions into a structured Boolean query and to analyze results for the most relevant documents from case law, legislation and commentary available on CanLII. 

A Boolean search query relies on a combination of words and phrases such as “AND,” “OR,” or “NOT,” as well as quotation marks and parentheses, to narrow down search results. While it is powerful and precise, it is not intuitive and most people who use CanLII do not necessarily know how to craft these queries.

In this episode of LawCast BC, we chat with Francis Barragan, President and CEO of CanLII and Emma Elliott, Vice President of Operations at Lexum, the firm that developed Search+. They explain what problems the tool solves, how it actually works and how it differs from other AI tools that are more prone to hallucinations or errors. We also walk through a couple of live demos with real-life prompts, demonstrating the power of the tool to analyze and score results based on relevancy. 

CanLII's entire technological platform is designed and maintained by Lexum, a legal technology firm that is owned by CanLII. Lexum continuously improves upon CanLII’s software infrastructure, including with the creation of Search+. 

Resources:

• Information about the CanLII Search+ research assistant
• Future updates about the Search+ tool can be found on the CanLII blog

Transcript

Vinnie Yuen:

Welcome to LawCast BC, a podcast produced by the Law Society of British Columbia. The Law Society regulates lawyers in BC. Our mandate is to protect the public. I'm Vinnie Yuen, host and producer. 

It seems that AI is everywhere. Companies are increasingly introducing AI into their workflows. According to a study commissioned by IBM, 79% of Canadian office workers are using AI tools at work. Whether we like it or not, it seems that AI is here to stay. One organization is harnessing the power of AI to help Canadians with legal research. If you've ever had to look up a legal case, whether you're a lawyer or just someone trying to figure out something for your own legal issue, you know that legal research can be quite challenging. CanLII recently launched a new artificial intelligence research tool called Search Plus to make access to legal information more intuitive and efficient. 

You can ask your question directly in plain language, and the tool instantly converts it into a structured query to target the most relevant documents from available case law, legislation, and commentary available on CanLII. For those of you who may not be familiar with CanLII, it's short for Canadian Legal Information Institute. It is a non-profit, free website that provides open online access to judicial decisions and legislative documents. It's funded by the Federation of Law Societies of Canada on behalf of 14 law societies, including the Law Society of BC. Joining us today is Francis Barragan. President and CEO at CanLII, and his colleague Emma Elliott, Vice President of Operations at Lexim. They'll walk us through how the Search Plus AI project came about, what problems they aim to solve with the tool, how it can search and analyze results, and will also run through a couple of live demos of some real-life prompts to input into the tool. Here's our discussion. 

Welcome, Francis and Emma. Thank you so much for joining us on our podcast. We're very happy to have you here today.

Francis Barragan:

Thank you for having us.

Vinnie Yuen:

Could you both tell us a little bit about yourself and your role at CanLII?

Francis Barragan:

Yes, of course. So, my name is Francis Barragan. I'm a lawyer from Quebec, and I've been the CEO of CanLII since September 2022. I come from the access to justice world, public legal education. I've been doing that for many, many years, so I really do believe in better access to justice, and that's why I love being at CanLII.

Emma Elliott:

For myself, my role is VP of Operations at Lexim. So, Lexim is a company that is fully owned by CanLII. We are CanLII’s main technology providers and have been so for 25-odd years. I am also a lawyer from Quebec. I've been at Lexim for 15 years, and I've been VP of Operations for the last five years. I'm essentially responsible for the delivery of our projects, services, and the development of our legal research products.

Vinnie Yuen:

Wonderful. Thank you so much. We're very excited to see that the Search Plus AI tool at CanLII launch. It's a project supported by the Federation of Law Societies and law societies across Canada. Why did CanLII decide to create the Search Plus AI tool? And how did the project fit into your mandate as an organization?

Francis Barragan:

So, the original mission of CanLII, which by the way, was launched 25 years ago this year, was to make the law available for free on the Internet. And that's what we've been working with in the last 25 years, trying to make law accessible. So, but when we define what is available, what does it mean to make something available? I think we've been long past the idea of, you know, you just put something on the web, and there goes your responsibility. It's much more than that. You have to be able to find the data, to be able to play around and find what you need for your purposes, right? So, we've been working at CanLII and Lexim for many, many years on how to have the best search engine that we were able to have. Refinement of the search engine has been a tremendously big project for many years. So, of course, when AI hit the, what I would call the mainstream market, when we started to talk about ChatGPT. Well, it's great with text, it's great to finding text, it's great to understand text. The link between what CanLII does and what AI knew AI possibilities were, I mean, it was a pretty straightforward fit. Now we had to make it work within what we're trying to do as well, of course. Not trying to have tools that are going to solve every possible problems but really make it fit within our mission. 

Now, when we started, we started to talk about this more openly at the beginning of March 2023 with our federation stakeholders, our law society stakeholders. They really helped us to try and find the right fit and how we're going to make this work for the benefit of the public. And that's why we were able to get the support to create this tool. Obviously, we think it improves access to the law, access to the primary authorities, but also access to justice in general. We haven't been making promises that we can't keep. We're very transparent about what it does, what it cannot do, and how it does it. This new tool will keep the primary law, the authoritative source documents at the very center of the tool. So, all those things fit within the mission that we have and that we had.

Vinnie Yuen:

What goals were you aiming to achieve with the creation of Search Plus? What problems were you trying to solve for the user?

Franic Barragan:      

When CanLII was created, if we go back 25 years ago, legal professionals from that time will probably remember how much it cost to get primary law. You would have to pay to get some of the case law because it had to, you know, it was, at the time, the way things were. When CanLII arrived, the idea was let's put it available to all on the internet. So, we wanted to make sure that we were able to create tools that would be able to be used by all and not be behind a big moat as it was 25 years ago, so that only people that could get the benefits of these new technologies would be by subscribing to very expensive tools. Our goal is trying to bring to the mainstream these tools so that everybody can at least be able to do research with new technological tools. It's easy to add new things, to add new features to a product, but I think you need to sometimes take a step back and see, well, why do people use your tools? 

What do they like about it? What do they like about what you're offering? And when you look at CanLII, CanLII receives about 21 million visits per year. That's huge for a very specific website, right? You go to CanLII if you're a legal professional or if you have a legal issue. So, we do have a lot of traffic, a lot of people, that consult and use our website. So, why do they use it? Well, because it's easy to use, because it's fairly fast. We think it's really fast. It's rigorous and trustworthy. People trust it. We're not trying to upsell anything. We have a core database that we want to help people find what they're looking for within that database. So, that's what people love. It's easy, it's fast, it's there when I need it, and I know I can trust it. So, that's what we try to apply with the creation of Search Plus.

Emma Elliott:

I would reiterate as well, as you were saying, we have a very large database. We're close to 4 million documents, and it's not everyone who is a seasoned legal researcher. Not everyone is very comfortable with Boolean syntax. So for us, it's also really the goal is, I think CanLII recognizes, or we're very clear on the fact that our documents are the authoritative sources. We want to get people to the right documents, and this is a way to help them navigate in such a rich and comprehensive database. So, it really is going to be able to help to orient users who are unfamiliar with certain areas of law. How do you get to the right documents? What do you need to be spending your time reading to really answer your question and to help you out?

Vinnie Yuen:

I was curious that was part of the decision to create this project, was it maybe also just making sure that the expertise stays with CanLII? Because people might be using other AI tools, open AI tools to help them with their legal research, especially maybe people who are not trained in the law. Was that a part of the thinking, too? So, people are not using Gemini or ChatGPT to look for cases because of maybe at the risk of hallucinations.

Francis Barragan:

The idea is that people will use ChatGPT and they will use Claude or they will use all kinds of tools. And I mean, we can't put our head in the sand. And what we're trying to say is, in our mindset is, we have this list of authoritative sources and we're going to help you as much as we can to be able to find what you need. But, you know, ChatGPT and other, those providers, well, they're still going to be used by people. But hopefully, when what we're trying to do is when you need to go deeper, and you want serious answers, and you want to be sure that you're not in trouble, you're going to have this source that's going to be useful for you.

Emma Elliott:

Our system is really grounded in CanLII’s database. So, the system is not searching the open web. It really is, anything it generates will be based on the source documents. And as I mentioned, being able to always link to that source document is another safeguard that really makes us, it makes sure that the system is based within the source document versus external information.

Francis Barragan:

And the tool cannot invent a decision because that's not how it's created. It's created to actually look at those decisions and tell you which one is the good one, but it can't create something out of thin air in that context.

Vinnie Yuen:

Yeah, I think that's very comforting for people to know because, you know, we see hallucinations in the news a lot. I think it's really comforting to know that this Search Plus tool will always bring you to the source documents. What are the main functions of Search Plus and if you could also let us know what are its limitations?

Emma Elliott:

I'd say in a nutshell Search Plus really is to help ask legal questions in plain language. So, the first thing that is new within the CanLII search environment is that you can now enter a plain language question into the research assistant. You'll find relevant sources. Based on that plain language question, we're going to, there's a functionality, the button is search. We'll generate a Boolean query for you. So, essentially, will translate that plain language question into language that the search engine can understand. And then we offer the possibility – of course, when we go ahead and do the demo, you'll see that we always provide the reasoning. You can see the different queries that the system has tested out. You can decide to say, well, I would like to try out another query instead, versus the one you've proposed. So, there is a lot of transparency built into the creation of those Boolean queries. Once you have been able to search and whittle down your results, you can then analyze. So, the other functionality is we will, based on the question you've asked, how does this document, how do these specific documents answer that question that you've asked? We'll provide a ranking as well, so from zero to five. The best 25 results we found for you will be analyzed. 

You can then re-rank, for example, in order of best result to lowest, and we'll provide an analysis where exactly we will give users the ability to understand why this document is pertinent to their question. This is really the portion that helps users identify relevant documents and understand how they relate to them. So, essentially, giving them a good indication that they should go and read this specific document or a specific paragraphs. For example, one of the things that the tool does offer as well is, whenever possible, we'll point users directly to the section or the paragraph where the system is getting its content from. When it comes to limitations, I would say, like any generative AI tool, there is a risk of errors and inaccuracies. So, we do need to be mindful of that. We are not providing legal advice and verification is always required. So, the original legal documents are really the authoritative sources, and it's up to the user to consult and to really, to be sure that they're relying on appropriate results. I would also say, as another limitation, technically speaking, we have daily quotas. So, for the moment, users are able to do up to – to generate up to 10 searches and do up to 100 analyses, so four batches of 25 results.

Vinnie Yuen:

Now that you've talked about the functions of the tool and some of the limitations, I'm wondering if you could walk us through a live demo and then maybe if you could give us a few examples. Just a note for those of you who are listening, we welcome you to open up CanLII on your browser to follow along for this demo.

Emma Elliott:

OK, when you land, arrive on canlii.org, the search field have not changed. You can absolutely decide to interact as usual with your databases or navigate or find information in any ways you were used to beforehand. You would see a login link at the top right-hand corner of your screen. That is what will allow you to log into your My CanLII account. So, all of the Search Plus functionalities are available for free once you've created that account. So, you'll see that the Search Plus functionality is available on the right-hand side of the screen. There's a little expand functionality. When I go and click on that, I have a menu that now appears on the right-hand side. There are some tips and tricks that are provided, a warning that's very much in line with the things we've discussed. Now that I'm ready to enter in my plain language question, I –

Vinnie Yuen: 

Yeah, maybe just describe the situation, like who are we pretending to be right now and then what we're looking for.

Emma Elliott:

Perfect. So, the question we're using at the moment would be a question I think would be typical of a public user. So, a BC citizen who is wondering if a landlord can terminate a tenancy because a tenant is making excessive noise. So, you're having trouble with your landlord. What do I do? Do I need to be worried of the threats I'm receiving? That I might be kicked out because I'm making excessive noise. So, I've gone ahead and entered my plain language question. I'm going to go ahead and hit the search button. And you'll see there's a pacifier that appears, and CanLII is thinking about our question. You'll see that there's also a show more, expand functionality. That is where, as I mentioned a little bit earlier, we'll provide the reasoning. We'll see the different queries that the system has tested out. There's an explicit explanation given as to why the system has chosen the query that it will use to populate the search. And then, so automatically, the system has launched that search. So, essentially, I have quite a sophisticated Boolean query that is presented. In this case, we can see that the BC jurisdiction has been selected automatically. So, the system knew to limit searching within British Columbia.

Vinnie Yuen: 

Yeah, so, just to summarize for our listeners, what Emma typed in was in BC, can a landlord terminate a tenancy because the tenant is making excessive noise? And then it has transformed that question and has turned it into a search that includes, you know, with brackets and quotations, “Residential tenancy act or RTA, end tenancy or terminate, and then noise or disturbance.” So, all of those words are now part of the search query that is formed based on that question.

Francis Barragan:

I was just going to add that if you look at the query that the system has created, now obviously it has used legal terms because the prompts and whatever, all the engineering behind it is to help to translate also in legal terms. So, in this case, it has identified we're talking about probably this act or this act. And when we talk about excessive noise, well, it could be noise or disturbance.

Emma Elliott:

Absolutely. So, shall we move on to analysis? 

Vinnie Yuen:

Yes. 

Emma Elliott:

Analysis functionality. Perfect. So, now that we have our satisfactory search query, I'm going to go ahead and click on analyze. So, now we'll see, again, different pacifiers showing us the progress the system is reading, analyzing, scoring, and summarizing. And we will end up with 25 ranked results. So, there's a little logo that appears where we'll see that they're ranked from zero to five. I'm going to go ahead and select rank by Search Plus score. So, instead of scrolling through my 25 results, I'll ask to see the most pertinent ones first. Displayed with every search result, so the link to the decision, the neutral citation, and so on and so forth, there is now a section called relevance analysis. And this really is the portion where we explain, or the system explains, why is this document pertinent to my question. So, for example, in the decision, Lee versus Wiedekind, a judicial review of RTB decisions upholding a landlord's one-month notice to end tenancy for cause based on persistent loud noise and unreasonably disturbed under a section of the RTA. Following that short analysis, we have links to the specific paragraphs where the answers to the question will be found more specific. So, in this case, I have one answer that's ranked five on five, I have a few four on fives, three on fives, and so on and so forth. So, and again, what's nice as well is it'll tell me why a document is relevant, but as we get to something that is less relevant, it'll also tell me why it's less pertinent to spend time reading that decision as well.

Vinnie Yuen:

We have a really relevant case here, five out of five. What would be a zero out of five? Maybe if you could just highlight?

Emma Elliott:

So, we have a decision that is ranked zero on five, and it is because this case addresses a BC landlord's one-month notice to end tenancy for cause, but the cause is hoarding and cleanliness, and not excessive noise. So, again, related to the termination of a tenancy, but it's not because of excessive noise. So, we know that this is not a decision that will apply well to our case.

Vinnie Yuen:

Yeah, so even though those keywords pulled up, this is one of the results, the Search Plus tool is able to analyze that and say, even though those terms are in here, it's not relevant to the question that you're asking.

Emma Elliott:

Correct, exactly. And you put your finger on it. The document was retrieved because exactly that, it was able – it contains all those terms from our Boolean search. However, the system can say, though it contains those terms, it's not really going to help answer your specific question.

Vinnie Yuen:

That's so handy.

Francis Garragan:

And then afterwards, the first decision that you see, then you can go click on it, right? And then all the other tools from CanLII are going to be there as usual, right? So, you're going to be able to see the cited documents, the treatment, if there's any, history. And also, if you start with the analysis, you can start right at the good paragraphs that you should start with. So, it really cuts down on, we believe anyway, it cuts down on trying to start your search.

Vinnie Yuen:

Great, if we could do, yeah, one more example from a different perspective, that'd be great.

Emma Elliott:

Perfect. So, for, let's see, I guess let's do from a lawyer's perspective. So, I have a case, let's see, my question is, so what cases discuss the limits of Tran versus Canada, which is a Supreme Court of Canada decision from 2017. So, again, as a first step, I'm going to go ahead and click on search and have the system propose a search query for me. In this case, we've been able to get down to nine relevant cases. So, again, the system has automatically populated my search with a sophisticated search query. I won't read through all of it. There's quite a few terms, but essentially, we're looking for the neutral citation. We're looking for the case name. We're also looking for decisions that will say distinguished, or limited to, or confined, or does not apply. And again, automatically, the system has brought us directly to the cases tab. So, the system knows we specifically asked for cases, so now we're no longer searching in legislation or commentary. And in this case, so again, I'm going to go ahead and analyze. Once again, I have one result that is five on five, one four on five. a three and a couple of zeros. So, already if I take a look at the first result that is ranked five on five, the decision R versus Archambault, which is another Supreme Court of Canada decision. 

This decision distinguishes Tran and sets limits by saying that Tran’s interpretation cannot be uniformly transposed to the Criminal Code Section 535 because of different text, context, and purpose. So, again, the system is explaining to us how these specific decisions are discussing the limits of a Supreme Court of Canada decision. And once again, we have our links here to specific paragraphs that will help us analyze and then land within our decision to be able to, again, as Francis mentioned, exploit all of CanLII’s rich tools, the search, highlight functionality, cited documents, AI analysis, and so on and so forth.

Vinnie Yuen:

Great. Yeah, thank you so much. I found that really helpful. How do you see Search Plus making a difference for self-represented litigants or anyone like myself who does not have a legal background?

Francis Barragan:

First things first, even for legal professionals, legal research is not always top of mind. You have legal researchers that do a lot of legal research every day, and you've got lawyers or other legal professionals that do legal research once in a while. Of course, I think the easiest way to see it for self-represented litigants or anyone not trained is it's going to be less noise. It's going to put the focus back on the documents themselves. When you go in front of a judge, and you want to prove your point, you have to have a basis in law. You have to have a document. You have to have something to work with. And that's what we're aiming to do, to bring it back to those documents. Here is the contents that you need to look into to help your case. So, less noise. Obviously, helping people that don't know the legal words. And again, that includes lawyers as well.

Vinnie Yuen:

I also was curious that part of the description for Search Plus is it says it's designed to support human intelligence and not to replace it. If you could tell us a little bit about what that means and what the thinking was behind that.

Francis Barragan:

As far as supporting human intelligence, well, I think there's a big amount of examples out of mainstream media or legal news or whatnot, where the tools get it wrong. The tools get stuff, complicated things wrong. The tools get simple things wrong. They got a lot of things right. But then where are you in this, right? So, what we're trying, what we've designed, is at the very start of this, you're in control of what's going to happen. You're in control of the documents. You're in control of the search. When you look at some of those answers that you'll get on ChatGPT, they're very convincing. This is the way, this is the thing. Here we're saying, listen, look at this. This is what it says. We believe it answers your question in this matter. Go look at it. You shouldn’t cite whatever we have created. So, you're fully in control as a person because that's where your worth is, and I believe will be for a long while. One of the best feedback we got or feedback metric that I was happy to see is the amount of people that says well, yes, it did save me time. Because that's the goal. It's going to save you time. It's not going to do everything for you. We’ve never promised that it was going to do everything. It's going to save you time, and hopefully a lot of time. And maybe it'll make you a better researcher, hopefully, when you see how it thinks, and maybe then you can make changes, and it's really as far as you want to take it.

Vinnie Yuen:

And if people want to learn more, or you mentioned maybe there might be updates, as more users have used the tool and give feedback, where can they look for those updates?

Francis Barragan:

So, we're very, we're trying to populate, of course, our LinkedIn feed, our CanLII blog, where we talk with our users. That's where we put our main announcements. But we also have fantastic relationships with the law societies across Canada. So, you can hopefully also look into the communication tools of your law societies. And so far, the Law Society of BC has been fantastic in announcing CanLII Search Plus in many different ways. One thing that I will add before we end this conversation is we are always happy to hear feedback from our users. That's how we try to create things that are helpful to our users. And that then can take many forms. It can go through Search Plus and how they use it, but also what I hear often, “There's a decision that I haven't found. I know it exists.” You can talk to us through feedback and say I'd like to have this decision be put on CanLII and it happens, we'll go look for it and ask it of the court. So, there are many ways that users can help us out and those are two.

Vinnie Yuen:

Wonderful. Thank you so much. Thank you, Emma and Francis, both of you, for your time today. And thank you for talking to us and walking through the tool with us.

Francis Barragan:

Thank you so much for having us.

Emma Elliott:

Yeah, likewise. It was wonderful to meet you and live this experience with you guys.

Vinnie Yuen:

Thank you for listening. To find out more about CanLII, please visit their website at canlii.org, spelled C-A-N-L-I-I.org. I'll also be including some links to their blog and information about the CanLII Search Plus tool in the show notes.

[Podcast ends]