In this episode of the Contract Heroes Podcast, our hosts Pepe and Marc speak with Otto Hanson, founder of TermScout and Screens and current leader of Screens at Agiloft. Before he founded those businesses, he worked as an attorney in a Finance and Acquisitions group, giving him a unique view into several sides of contract management. In this episode, Otto shares his insights into common concerns about using AI for contract review, how Screens empowers contract management teams, and the future of AI in CLM. You can hear all of the details by listening to this episode of the Contract Heroes Podcast or read on to get a quick overview of what he spoke about.
Meet Otto Hanson: Founder of TermScout and Screens, Leader of Screens at Agiloft
Otto has been working on the pre-signature contract review problem since the inception of Screens, which was founded in July 2023 and acquired by Agiloft in December 2024. Before generative AI, they had trouble building a review product that was both accurate and flexible: it was always a tradeoff of one or the other. With generative AI, however, this tradeoff ended, and new tools empowered them to build pre-signature contract review tools that made it easier for anyone to review contracts and understand their main points. Especially since ChatGPT’s launch, Otto has been focused on building better and better tools to help improve contract review processes for businesses of all sizes.
Concerns About AI Hallucinations and Contract Review
It’s important to consider what use cases you’re looking at for AI features. With contract review, you are at a relatively low risk for hallucinations because you aren’t asking generative AI to create something completely new or inform you on a topic you haven’t previously shared information about. Instead, you’re having it work with information that you are providing, either by sharing it in a prompt or uploading a document. Then, you are asking it to deliver answers to specific questions about the content you have provided.
Because of this system, it is also relatively easy for you to verify the information that an AI feature provides. Within a single click, you should be able to get from AI’s response to the exact language in the document that it is referring to, where you can verify whether its conclusion is correct.
With Screens, you can also have confidence in their rigorous auditing for accuracy. They gather an evaluation data set that includes several different types of contracts and create standards that lawyers usually prioritize, such as limiting liability or what happens in case of a security breach. Then, they have human lawyers review the contracts using these standards before plugging them into large language models so they can compare the results and select the most accurate LLM for customers. With this system, they have achieved a 97.5% accuracy rate for standard identification tasks and whether redlines that the platform proposes fix the problems that they are meant to fix.
How Screens Helps Users Create Powerful Prompts
When users are reviewing contracts in Screens, they are prompted to rate each response with either a thumbs-up or thumbs-down. After 20 results, depending on the percentage of thumbs-up vs thumbs-down responses, Screen will prompt users to tweak their prompt to get better results. This feedback loop then repeats to further optimize prompts and ensure that users are able to get consistent results that are focused on what matters most to them. Agiloft also offers the GenAI Prompt Lab to help users further fine-tune their prompts and improve their results.
Improving User Experience By Combining Screens with Agiloft
Otto is bullish on the CLM space, and he has several reasons to feel that way. First and foremost, contracts are complicated and each one will have its own quirks. As a result, generic solutions that are meant for handling all types of documents or customer relationships are unlikely to solve all of the problems that contract management teams have in their workflows. Instead, contract lifecycle management requires software that caters to the unique nuances and concerns of contract management teams and allows them to use generative AI to be as efficient as possible.
Before the acquisition, Screens was primarily focused on the pre-signature contract review. However, their clients needed additional services that Screens was not going to be able to provide, which is where Agiloft comes in. Because Agiloft already had an expansive and versatile CLM solution, it made sense to pair Screens’ excellent pre-signature contract review solutions with their existing set of features to help clients of both companies experience a smoother, more efficient contract management solution.
Combining these tools also empowers users to further automate the contract review processes they’ve been using Screens for. For example, if Screens identifies a standard that is not met in a contract, Agiloft can automatically message the stakeholders who need to be involved in updating that contract so the standards will all be met.
AI’s Role in Knowledge Work and Legal
One of Otto’s insights from this episode is that he believes AI’s best use case in the knowledge workspace, and especially in legal, is not to make nuanced decisions like what should or should not be included in a specific contract. Instead, it is meant to help professionals scale their expertise and perform basic administrative tasks that are often repetitive and burdensome to do manually. Human experts will always be the best and most trusted advisors, but AI can help those experts to do their job more efficiently.
The Screens community is a place where experts can share their expertise about what should always be within a specific type of contract, and AI can take that expertise and apply those standards to hundreds or even thousands of contracts with very little additional effort from human experts. This way, they are able to manage the administratively difficult tasks without attempting to make contract management decisions that really demand custom expert insights.
Another feature that is a customer favorite is when Screens catches a one-sided clause that needs to be made mutual before the contract is signed. Screens’ AI tools can identify those clauses and make suggested edits, which human experts can then decide to accept, modify, or reject.
Using AI Tools for Your Contract Management Processes
AI won’t always get these things perfectly right, but it’s so accurate that it still meaningfully reduces the workload on experts and empowers them to be more efficient. Otto expects that AI tools will continue to help professionals become more efficient to ensure that all the legal work that needs to be done is done, with minimal stress on the human beings behind the screens. While many professionals may be fearful of AI eventually replacing them, Otto’s experience has shown that, instead, AI can make contract management professionals’ work easier so they can get more done in less time.
If you aren’t sure whether your business is using contract management AI tools to their full potential, consider reaching out to Koho Consulting, who sponsored this episode of Contract Heroes. Their team can help you make the most of your CLM at every stage, from finding the right software to maintaining your existing systems.
