Applied Economics: Business, Personal and Complex Litigation: When Steven Sanchez was a senior at Stanford University he had two forward paths from... How to Protect Your Private Practice From a California Payroll Tax Audit: Did you know you can be held personally liable for payroll taxes if the California... Are You Suffering From Solo / Small Firm Imposter Syndrome?: Until recently, practicing law in the private realm pretty much came down to two... The Lateral Attorney Transition Two Tips To Make Your Next Hire a Good One: Anyone in the legal community knows that a popular method for growing business is to... 6 Reasons Why Content Is too Important for Lawyers to Write: Content is about the music, not the words. That’s why it is too important to be written... California Case Summaries ADR™ Organized Succinct Summaries of New California Civil Cases: CALIFORNIA COURTS OF APPEAL - Arbitration - Uber Technologies v. Google (2018) _... Amp up Your LinkedIn Profile With These 10 Tips for Lawyers: The new year is approaching fast and amidst the last-minute holiday shopping, you can be... Community News – December 2018: Vanst Law recently opened its virtual doors. At this modern law firm, all attorneys are... From Confusion to Clarity for Dentists and Physicians: San Diego’s Attorney for Dentists and Physicians Earns a Position on the Leading Edge... 5 Tips for Young Lawyers How to Find Your Way in a Crowded Arena: Young lawyers can face some uphill battles out of law school. The dream of defending the...
Executive Presentations-468x60-1

7 Practical Suggestions to Get the Most out of Technology-Assisted Review

Virus and hacking concept

With the recent high profile use of technology-assisted review in the Da Silva Moore and Kleen cases, and with positive judicial opinions coming from Judge Peck and others, many legal teams are considering using technology-assisted review for the first time. Without expert guidance, newcomers frequently struggle with the tools and processes of technology-assisted reviews. Here are some real-world guidelines for achieving the best results.

The basic underlying technologies have been around for decades. Machine learning techniques are used in a wide variety of industrial applications, including speech recognition, facial expression categorization, handwriting recognition, and computational biology, to name a few. Applied to the problem of large-scale legal document review and production, technology-assisted review enables a small number of expert human reviewers to train a computerized system to recognize the subtle differences between responsive and non-responsive documents based on the review of a small but representative sample of the document collection.

With explosive growth in electronically stored information, a system that magnifies the efforts of a single reviewer (such as assisted review) is seen as an important step towards increasing electronic discovery quality while simultaneously reducing cost and time.

To take best advantage of this new technology, here are some practical tips based on our experience in supporting dozens of real-life cases.

1. Use subject matter experts to train the system. Technology-assisted review magnifies the decisions made by the reviewers training the system. For consistency, the same set of expert reviewers should train the system from start to finish. This is something that can be delegated to outsourced review management in some cases. For example, a case involving an intellectual property dispute related to biotechnology may require experts in that field to determine which documents are relevant.

2. Use a review manager that understands technology-assisted review. For large cases, law firms and corporations often rely on outsourced review management. The review management team is often in the best position to use assisted review most effectively. Assisted review can be used in different ways depending on the specific needs of a case. These include prioritizing document review, using flexible staffing models to distribute review, bulk classification of non-relevant documents, and early case assessment, to name a few. Before starting, plan out how the technology and associated workflow fit into the electronic discovery plan for a given project. Having a review manager and a separate assisted review team can reduce the possible benefits if these teams don’t work closely together. Sidestep the issue by using a review manager well-versed in the assisted review process employed.

3. Learn the terms. There are several words frequently used in technology-assisted review that have a distinctive meaning. Richness, recall, precision, and f-measure are four key terms to know. Richness is the overall percentage of the document population that is responsive to an issue. Recall is the percent of the responsive documents identified by the process and is a measure of completeness. Precision is the percent of documents identified by the process as responsive that are actually responsive and is a measure of quality. F-measure is the weighted average of precision and recall. There are many more to learn, and knowing the vocabulary helps.

4. Apply technology-assisted review before culling with search terms. Using Boolean search terms to cull down the population ahead of assisted review can reduce the overall recall potential. A limitation of Boolean search is that you often find only what you know to look for. Using Boolean search requires the case team to know exactly what language was used in the target documents ahead of time. The strength of technology-assisted review is that it can uncover pockets of relevant content that were not suspected. Most assisted review processes sample across the document universe.

5. Use blind sampling to measure progress and avoid reviewer bias. Many technology-assisted review processes build in sampling to measure progress and provide quality control throughout review. When reviewers are aware that a document is marked responsive or not responsive, it can influence their decision. Representative sampling is a well-established method of determining the characteristics of a large population of documents.

6. Focus on one or two issues at most. Assisted review benefits are highest when ranking documents against the primary issues at the heart of a matter. There are generally dozens of sub-issues. Subdividing responsive documents into the many subcategories is best left to second pass reviewers with explicit review instructions and training.

7. Use second pass linear review. Use a second pass review to identify additional non-responsive documents as well as to identify documents to be withheld for privilege.

Technology-assisted review promises to reduce cost and improve quality compared to other available methods. With the exploding growth of electronically stored information, these new tools are needed. The legal teams that master them first will have a great advantage.

Jon Lavinder

Jon Lavinder, director of technology-assisted review at DTI.

More Posts

Digg This
Reddit This
Stumble Now!
Buzz This
Vote on DZone
Share on Facebook
Bookmark this on Delicious
Kick It on DotNetKicks.com
Shout it
Share on LinkedIn
Bookmark this on Technorati
Post on Twitter
Google Buzz (aka. Google Reader)
www.pdf24.org    Send article as PDF   

Filed Under: Technology

About the Author: Jon Lavinder, director of technology-assisted review at DTI.

RSSComments (0)

Trackback URL

Leave a Reply

  • Polls