LLM-driven document analysis validated the accuracy and efficiency of our methodology while enhancing comprehensiveness.
The Data Science Center leveraged large language model (LLM)-driven document analysis to efficiently validate the accuracy of our results based on traditional methods, in a corporate governance matter.
The Challenge
Our team sought to identify instances in over 1,000 documents where a particular relationship among affiliated companies was alleged. After pre-processing, the LLM was prompted to review all discussions surrounding the companies, without any other keyword constraint. The LLM flagged just 15 pieces of content for manual review. Consistent with the traditional review process, the LLM-assisted analysis supported the non-existence of the relationship in question, with greater comprehensiveness and confidence.
Key Takeaways
- Enhanced Coverage: The LLM-driven approach enables a more comprehensive review by allowing for analysis of a much larger set of data using fewer restrictions than traditional methods (such as specific keywords), thus reducing the risk of missing critical information.
- Accelerated Analysis: For matters with tight deadlines, the LLM-assisted analysis can make previously infeasible analysis possible. In this instance, the LLM processed more than 100,000 paragraphs of text in approximately one hour, unlocking valuable insights.
- Flexible Integration: The LLM-driven method is compatible with and seamlessly integrates with traditional review. The LLM can either serve as a first-pass “doer” to prioritize the review of keyword search results or as a “checker” to enhance comprehensiveness and accuracy.