A Specialized Personal Injury & Mass Tort Trial Firm. Getting Extraordinary Results for Ordinary People: Creating the Right Team to Take on Select Cases that Excite Attorneys, Right Wrongs and... Optimizing Your Law Firm’s Financial Health in 2019: Four Areas to Focus Your Spring Cleaning: Fingerprints are unique. No two snowflakes are alike. And each law firm has its own... How to Find and Keep Talent in a Highly Competitive Market: One of the biggest problems companies face, in particular middle market companies, is a... Lessons Learned from Navy SEALs: I will admit it up front—I am in awe of Navy SEALs. My brother-in-law was a career... To Speak Like the Best, You Need to Prep Like the Best: I recently went to the Legal Marketing Association’s (LMA) educational program,... Overconfidence and Law Firm Decline: “Every institution is vulnerable, no matter how great. No matter how much you have... Making a Move with a Purpose: Three to five years into an associate’s legal career is the prime time for associates... Snap! And You Miss It: Snapchat and other photo-sharing apps have redefined how users and employees share... Nobody Wants a Debbie Downer During Initial Consultations: Initial consultations are an important part of most law firms. Potential clients meet... Community News – April 2019: Fifteen Wilson Turner Kosmo attorneys have been selected for inclusion on the San Diego...
Executive Presentations-468x60-1

7 Practical Suggestions to Get the Most out of Technology-Assisted Review

Virus and hacking concept

With the recent high profile use of technology-assisted review in the Da Silva Moore and Kleen cases, and with positive judicial opinions coming from Judge Peck and others, many legal teams are considering using technology-assisted review for the first time. Without expert guidance, newcomers frequently struggle with the tools and processes of technology-assisted reviews. Here are some real-world guidelines for achieving the best results.

The basic underlying technologies have been around for decades. Machine learning techniques are used in a wide variety of industrial applications, including speech recognition, facial expression categorization, handwriting recognition, and computational biology, to name a few. Applied to the problem of large-scale legal document review and production, technology-assisted review enables a small number of expert human reviewers to train a computerized system to recognize the subtle differences between responsive and non-responsive documents based on the review of a small but representative sample of the document collection.

With explosive growth in electronically stored information, a system that magnifies the efforts of a single reviewer (such as assisted review) is seen as an important step towards increasing electronic discovery quality while simultaneously reducing cost and time.

To take best advantage of this new technology, here are some practical tips based on our experience in supporting dozens of real-life cases.

1. Use subject matter experts to train the system. Technology-assisted review magnifies the decisions made by the reviewers training the system. For consistency, the same set of expert reviewers should train the system from start to finish. This is something that can be delegated to outsourced review management in some cases. For example, a case involving an intellectual property dispute related to biotechnology may require experts in that field to determine which documents are relevant.

2. Use a review manager that understands technology-assisted review. For large cases, law firms and corporations often rely on outsourced review management. The review management team is often in the best position to use assisted review most effectively. Assisted review can be used in different ways depending on the specific needs of a case. These include prioritizing document review, using flexible staffing models to distribute review, bulk classification of non-relevant documents, and early case assessment, to name a few. Before starting, plan out how the technology and associated workflow fit into the electronic discovery plan for a given project. Having a review manager and a separate assisted review team can reduce the possible benefits if these teams don’t work closely together. Sidestep the issue by using a review manager well-versed in the assisted review process employed.

3. Learn the terms. There are several words frequently used in technology-assisted review that have a distinctive meaning. Richness, recall, precision, and f-measure are four key terms to know. Richness is the overall percentage of the document population that is responsive to an issue. Recall is the percent of the responsive documents identified by the process and is a measure of completeness. Precision is the percent of documents identified by the process as responsive that are actually responsive and is a measure of quality. F-measure is the weighted average of precision and recall. There are many more to learn, and knowing the vocabulary helps.

4. Apply technology-assisted review before culling with search terms. Using Boolean search terms to cull down the population ahead of assisted review can reduce the overall recall potential. A limitation of Boolean search is that you often find only what you know to look for. Using Boolean search requires the case team to know exactly what language was used in the target documents ahead of time. The strength of technology-assisted review is that it can uncover pockets of relevant content that were not suspected. Most assisted review processes sample across the document universe.

5. Use blind sampling to measure progress and avoid reviewer bias. Many technology-assisted review processes build in sampling to measure progress and provide quality control throughout review. When reviewers are aware that a document is marked responsive or not responsive, it can influence their decision. Representative sampling is a well-established method of determining the characteristics of a large population of documents.

6. Focus on one or two issues at most. Assisted review benefits are highest when ranking documents against the primary issues at the heart of a matter. There are generally dozens of sub-issues. Subdividing responsive documents into the many subcategories is best left to second pass reviewers with explicit review instructions and training.

7. Use second pass linear review. Use a second pass review to identify additional non-responsive documents as well as to identify documents to be withheld for privilege.

Technology-assisted review promises to reduce cost and improve quality compared to other available methods. With the exploding growth of electronically stored information, these new tools are needed. The legal teams that master them first will have a great advantage.

Jon Lavinder

Jon Lavinder, director of technology-assisted review at DTI.

More Posts

Digg This
Reddit This
Stumble Now!
Buzz This
Vote on DZone
Share on Facebook
Bookmark this on Delicious
Kick It on DotNetKicks.com
Shout it
Share on LinkedIn
Bookmark this on Technorati
Post on Twitter
Google Buzz (aka. Google Reader)
www.pdf24.org    Send article as PDF   

Filed Under: Technology

About the Author: Jon Lavinder, director of technology-assisted review at DTI.

RSSComments (0)

Trackback URL

Leave a Reply

  • Polls