Quantifying the Benefits of AI-Enhanced Review
We’re bringing Smarter AI to Legalweek New York 2020
January 7, 2020
eDiscovery & Data Management in a Changing Regulatory Environment
March 12, 2020
Exploding data volumes coupled with high
costs have made the search for affordable, fast, and defensible eDiscovery technology
a top priority for legal professionals.
Before the big data explosion, legal case teams used to manually review every document produced in a proceeding. Today, that would be almost impossible and at best impractical; most litigators rely on a combination of technology and review teams for production.
But this is still a comparatively inefficient and error-prone process. Search parameters on their own can’t whittle down huge data repositories to manageable proportions, and the human element – an integral part of the process – can be pushed by the sheer scale of the task into risking quality as well as speed and efficiency.
Now, though, the big data phenomenon that’s causing these issues is also being leveraged to drive both human and technology-based solutions. Advances in AI have made it a force multiplier in document management. Coupled with behavioral science expertise, AI can now make efficient, high quality review possible while enhancing, rather than replacing, the human contribution.
AI-Enhanced review is an important step forward in the Technology Assisted Review (TAR) arena. , Our FRONTEO AI engine, KIBIT, extends this capability to make every stage of the review process more efficient and effective. KIBIT has the ability to pinpoint more relevant documents and can yield significant downstream savings by reducing data volumes sent for review – all while maintaining defensibility.
It has been gratifying for the FRONTEO team to see our clients use our newer approaches to achieve measurable gains compared with traditional linear review, and we have been excited to quantify these findings in a recent proof of concept.
To demonstrate the validity of these claims, we provided an AI-enhanced review Proof of Concept (PoC) to Jack Fornaciari, a BakerHostetler partner with a thriving antitrust practice. Mr. Forniciari is often faced with tight discovery deadlines requiring thousands of hours of document review. The data is frequently nuanced and sensitive, so review teams must consider each document carefully.
From one of Mr. Fornaciari’s prior cases, we produced simulated results using actual documents and review designations. We then staffed and conducted a full review to demonstrate how KIBIT increases human review speeds, efficiency and quality control throughout the duration of a live document review project. The results are documented in our white paper, ‘AI-Enhanced Review with KIBIT’, available for download here.
Like all AI-powered solutions, KIBIT is designed to augment human effort. This is accomplished not only by propagating attorneys’ decisions to similar documents, but also by facilitating an optimized workflow that accelerates review speeds and flags potential mistakes. We therefore fully replicated the human review component of Mr. Fornaciari’s matter.
The pace of document review tends to increase when similar documents are reviewed together. Attorneys can breeze through batches when coding designations are the same for dozens of documents in a row. With AI-Enhanced review, we use KIBIT’s relevancy score to create high- and low-scoring batches. Average review speeds tended to be slower for high-scoring batches that contained mostly relevant documents and faster for low-scoring batches containing more non-relevant documents.
In the PoC we were able to show how our targeted quality control (QC) ‘Heat Map’ gives Review Managers a visual representation of anomalies in reviewer coding. This allows them to better manage the quality as well as the speed of review. For example, team members moving too slowly can be a problem on an urgent review. On the other hand, an abnormally fast pace may be indicative of careless review and classification.
FRONTEO evaluated the PoC results via a number of relevant metrics and found that increasing the speed of review significantly reduced the overall cost of the project. Equally important, AI-enhanced review allowed for more accurate review of large data volumes within delineated timeframes.
Comparing the results, hybrid review yielded a pace of 73.7 documents per hour as opposed to 40 documents per hour with linear review. In combination with AI-only review, total review time was cut by 48%, meaning AI-Enhanced review was nearly twice as fast as linear review.
Finally, our PoC team was able to reduce the number of documents by 64% and achieved an average review speed of 91.6 documents per hour, cutting review time by 48% compared to the original linear review.
The PoC confirms that strategic deployment of artificial intelligence will result in ultra-efficient reviews while maintaining defensibility.
Jack Forniciari told us, “We substantially shortened the review timeline by integrating FRONTEO’s KIBIT without sacrificing quality. The results confirm the efficiency of using enhanced AI in document review and we look forward to utilizing it in future cases.”
At FRONTEO, we feel these results are very promising. They show that KIBIT aids human effort in a quantifiable way. Human error is greatly reduced, while speed and accuracy are amplified. This leads to greater efficiency for all stakeholders in a legal action, from defense teams to opposing counsel and judiciary.
We all have a stake in increasing the efficiency while lowering the cost of litigation, and that promise is being made real with AI-Enhanced document review.
Before the big data explosion, legal case teams used to manually review every document produced in a proceeding. Today, that would be almost impossible and at best impractical; most litigators rely on a combination of technology and review teams for production.
But this is still a comparatively inefficient and error-prone process. Search parameters on their own can’t whittle down huge data repositories to manageable proportions, and the human element – an integral part of the process – can be pushed by the sheer scale of the task into risking quality as well as speed and efficiency.
Now, though, the big data phenomenon that’s causing these issues is also being leveraged to drive both human and technology-based solutions. Advances in AI have made it a force multiplier in document management. Coupled with behavioral science expertise, AI can now make efficient, high quality review possible while enhancing, rather than replacing, the human contribution.
AI-Enhanced review is an important step forward in the Technology Assisted Review (TAR) arena. , Our FRONTEO AI engine, KIBIT, extends this capability to make every stage of the review process more efficient and effective. KIBIT has the ability to pinpoint more relevant documents and can yield significant downstream savings by reducing data volumes sent for review – all while maintaining defensibility.
Proving the Benefits
It has been gratifying for the FRONTEO team to see our clients use our newer approaches to achieve measurable gains compared with traditional linear review, and we have been excited to quantify these findings in a recent proof of concept.
To demonstrate the validity of these claims, we provided an AI-enhanced review Proof of Concept (PoC) to Jack Fornaciari, a BakerHostetler partner with a thriving antitrust practice. Mr. Forniciari is often faced with tight discovery deadlines requiring thousands of hours of document review. The data is frequently nuanced and sensitive, so review teams must consider each document carefully.
From one of Mr. Fornaciari’s prior cases, we produced simulated results using actual documents and review designations. We then staffed and conducted a full review to demonstrate how KIBIT increases human review speeds, efficiency and quality control throughout the duration of a live document review project. The results are documented in our white paper, ‘AI-Enhanced Review with KIBIT’, available for download here.
Like all AI-powered solutions, KIBIT is designed to augment human effort. This is accomplished not only by propagating attorneys’ decisions to similar documents, but also by facilitating an optimized workflow that accelerates review speeds and flags potential mistakes. We therefore fully replicated the human review component of Mr. Fornaciari’s matter.
The pace of document review tends to increase when similar documents are reviewed together. Attorneys can breeze through batches when coding designations are the same for dozens of documents in a row. With AI-Enhanced review, we use KIBIT’s relevancy score to create high- and low-scoring batches. Average review speeds tended to be slower for high-scoring batches that contained mostly relevant documents and faster for low-scoring batches containing more non-relevant documents.
In the PoC we were able to show how our targeted quality control (QC) ‘Heat Map’ gives Review Managers a visual representation of anomalies in reviewer coding. This allows them to better manage the quality as well as the speed of review. For example, team members moving too slowly can be a problem on an urgent review. On the other hand, an abnormally fast pace may be indicative of careless review and classification.
More Speed + Less Time = Lower Costs
FRONTEO evaluated the PoC results via a number of relevant metrics and found that increasing the speed of review significantly reduced the overall cost of the project. Equally important, AI-enhanced review allowed for more accurate review of large data volumes within delineated timeframes.
Comparing the results, hybrid review yielded a pace of 73.7 documents per hour as opposed to 40 documents per hour with linear review. In combination with AI-only review, total review time was cut by 48%, meaning AI-Enhanced review was nearly twice as fast as linear review.
Finally, our PoC team was able to reduce the number of documents by 64% and achieved an average review speed of 91.6 documents per hour, cutting review time by 48% compared to the original linear review.
Ultra-Efficient Reviews
The PoC confirms that strategic deployment of artificial intelligence will result in ultra-efficient reviews while maintaining defensibility.
Jack Forniciari told us, “We substantially shortened the review timeline by integrating FRONTEO’s KIBIT without sacrificing quality. The results confirm the efficiency of using enhanced AI in document review and we look forward to utilizing it in future cases.”
At FRONTEO, we feel these results are very promising. They show that KIBIT aids human effort in a quantifiable way. Human error is greatly reduced, while speed and accuracy are amplified. This leads to greater efficiency for all stakeholders in a legal action, from defense teams to opposing counsel and judiciary.
We all have a stake in increasing the efficiency while lowering the cost of litigation, and that promise is being made real with AI-Enhanced document review.
