LawGeex Wins AI v. Lawyers NDA Issue-Spotting Competition

On February 26, 2018, LawGeex announced the results of a competition for spotting issues in five (5) standard Non-Disclosure Agreements (NDAs) between its AI-based system and twenty (20) U.S-trained and very experienced lawyers. The competition was overseen by an independent consultant and lawyer, and involved input from academics, data scientists, and legal and machine-learning experts.

Bottom line (quoting from the Study report):  LawGeex "achieved an average 94% accuracy rate, ahead of the lawyers who achieved an average rate of 85%." (Boldface in the original.)

And it took the lawyers an average of 92 minutes to review all 5 NDAs, while the LawGeex AI reviewed all 5 NDAs in only 26 seconds.

So, LawGeex deserves a shout-out: Congratulations!

LawGeex has been on my radar since May of 2017, when it published its In-House Counsel’s Legal 2017 Tech Buyer’s Guide. LawGeex had placed Intraspexion in the Prediction Technology sector of a graphic LawGeex called the Legal AI Landscape. You can see the Landscape in my January post.

But here's why I was excited about the LawGeex "win." In the Study report. Professor Yonatan Aumann, who lectures in the Department of Computer Science at Bar Ilan University, described the AI that LawGeex had used. He said:

"The technology has been developed through a combination of supervised and unsupervised learning techniques. Unsupervised learning was used for teaching the AI engine the core legalese language. Thereafter, supervised learning, using deep learning multi-layer LSTM [n.b., the acronym for Long Short-Term Memory] and convolution technology, was used to train the system for the fine-tuned issue-spotting. Supervision was performed based on human-annotated documents, using legal experts. A unique augmentation algorithm was applied to boost learning from these examples." (Boldface added.) 

I had suspected that LawGeex was using deep learning, but had no confirmation. Deep learning is what Intraspexion uses.

Of course, there are differences, such as the fact that the LawGeex application is focused on contracts, while Intraspexion's application is litigation or, more precisely, surfacing the risks and providing an early warning to avoid litigation.

And Intraspexion uses a recurrent neural network, not a convolutional technology; and, because litigation categories already exist, Intraspexion didn't need the "unsupervised" step.

But LawGeex and Intraspexion are, nevertheless, "deep learning" practitioners and, in that vein, we are kindred spirits. We are tools for the Corporate Law Departments of the Future.

And, so to LawGeex, I say again, Congratulations!