Generative AI / LLM’s will not help lawyers draft contracts.
The legal industry has a deep-rooted tradition that has historically emphasized caution and precision — and for good reason. Deals can be complex, and mistakes can have severe consequences for clients.
AI Garbage in = Garbage Out
Most language models are built using a combination of text-data collected off the internet and those found in legal text (either publicly or privately) which serves as a foundation for the models training. GPT-4’s neural network was famously trained using 1 petabye (1 million gigabytes 🤯 ) of text from the internet and has 1.8 trillion parameters. Even with all that, GPT is prone to hallucinations and model degradation.
It’s probably safe to say that there’s nowhere near this amount of publicly available legal data which means there’s no way of creating as accurate of a model from the ground up. Instead, most companies in the space opt to use GPT or other language models as a base and layer in another fine-tuned model that utilizes proprietary legal datasets.
The output will be some combination of the larger base model and the fine-tuned legal one which introduces a few questions:
· Which model has more influence?
· What went into the base model and who oversaw filtering out bad data points? (There’s a lot of bad/false information on the internet folks.)
· How is the variability in the output controlled?
· Is my data being shared with other users?
Upkeep and Training?
Most folks have heard of model hallucinations by now but not many know about model degradation. This refers to the gradual decline in a model’s performance over time. The decline can be due to various reasons, including lack of re-training or changes in the user’s behavior which can lead to lower accuracy and poor output.
To address this, models will need support in the form of regular re-training and updates which require highly specialized teams that can continuously monitor, detect, and address the decay.
Bringing a team like this in house is just not economically feasible for most companies and the number experts on the subject matter are still low. Early adopters of 3rd party models might find that their license does not include a personalized service to handle this since it is hard to scale.
It’s still too early to determine how large of an impact this will have since the technology is still young, but I am already seeing companies offer staffing services for AI fact checkers, editors and even ethics advisors.
Contract Logic
Contracts are a lot like computer code. If you break the logic in one section of a contract, you are potentially impacting another.
Attorneys understand the invisible links between each of their provisions and work to ensure there are no conflicts in the contract the same way a software developer would check their code for bugs.
Language models unfortunately do not work this way. They generate text that is an amalgamation of the dataset it was trained on with very little relation or context to what it has already generated.
Don’t believe me? Ask GPT to write multiple articles and then combine them into one. The best I have seen here is a slight edit to the original output.
Any Real Time Savings?
All this really begs the question, is there any real efficiency gain for lawyers when it comes to using LLMs to draft contracts?
With all the time it takes to setup, review and ensure the accuracy of the output from a model, you might as well just draft the thing yourself. This is exactly why adoption has been so slow.
Lawyers need tools they can rely on and not babysit.
Intelligent Automation (A better way!)
Charta uses intelligent automation to ensure accuracy without the need for oversight or maintenance, and it takes drastically less time and resources to get up and running.
Contracts are drafted using your own provisions, so they look exactly the way you want them to. Best of all, the contract logic is correctly mapped using our patent pending process, preventing you from generating a broken document.
Interested in learning more? Reach out to us at partnerships@getcharta.com, on linkedin or via getcharta.com.
