Quarterbacking Success in a Challenging Field: Every team needs a quarterback, especially when that team is composed of attorneys,... Helping Clients Navigate a Challenging Legal System: “ We bring the same power,resources, experience, and novelapproach to every case we... How to Get More Law Firm Reviews Online: With all the stories we hear of identity theft and security breaches these days, it can... 5 Lessons for Law Firm Marketers From HubSpot’s INBOUND 2017 Conference: I know some folks within law firms are wary of attending marketing conferences outside of... Are You a Networking Rock Star or Rookie?: If you’ve read my book The Attorney’s Networking Handbook, you probably know that I... San Diego Personal Injury Attorneys Motivated to Care: “The most important thing is that you care about what you do. As an attorney I care... COMBINING HUMAN RELATIONSHIPS WITH TECHNOLOGY’S ADVANCES: Anthony Geraci, founder of Geraci Law Firm, believes the path to success in the coming... Networking: How To Disengage From a Dead-End Conversation: We’ve all found ourselves trapped in a dead-end conversation at a networking event.... What’s New in Structured Settlements? A Lot!: Structured settlements became a popular alternative to lump sum settlements in the 1970s.... Written Schedule of Services and Fees Multiplies Credibility, Increases Client Comfort: How do you present fees in a way that emphasizes the value of your services and takes the...
Executive Presentations-468x60-1

7 Practical Suggestions to Get the Most out of Technology-Assisted Review

Virus and hacking concept

With the recent high profile use of technology-assisted review in the Da Silva Moore and Kleen cases, and with positive judicial opinions coming from Judge Peck and others, many legal teams are considering using technology-assisted review for the first time. Without expert guidance, newcomers frequently struggle with the tools and processes of technology-assisted reviews. Here are some real-world guidelines for achieving the best results.

The basic underlying technologies have been around for decades. Machine learning techniques are used in a wide variety of industrial applications, including speech recognition, facial expression categorization, handwriting recognition, and computational biology, to name a few. Applied to the problem of large-scale legal document review and production, technology-assisted review enables a small number of expert human reviewers to train a computerized system to recognize the subtle differences between responsive and non-responsive documents based on the review of a small but representative sample of the document collection.

With explosive growth in electronically stored information, a system that magnifies the efforts of a single reviewer (such as assisted review) is seen as an important step towards increasing electronic discovery quality while simultaneously reducing cost and time.

To take best advantage of this new technology, here are some practical tips based on our experience in supporting dozens of real-life cases.

1. Use subject matter experts to train the system. Technology-assisted review magnifies the decisions made by the reviewers training the system. For consistency, the same set of expert reviewers should train the system from start to finish. This is something that can be delegated to outsourced review management in some cases. For example, a case involving an intellectual property dispute related to biotechnology may require experts in that field to determine which documents are relevant.

2. Use a review manager that understands technology-assisted review. For large cases, law firms and corporations often rely on outsourced review management. The review management team is often in the best position to use assisted review most effectively. Assisted review can be used in different ways depending on the specific needs of a case. These include prioritizing document review, using flexible staffing models to distribute review, bulk classification of non-relevant documents, and early case assessment, to name a few. Before starting, plan out how the technology and associated workflow fit into the electronic discovery plan for a given project. Having a review manager and a separate assisted review team can reduce the possible benefits if these teams don’t work closely together. Sidestep the issue by using a review manager well-versed in the assisted review process employed.

3. Learn the terms. There are several words frequently used in technology-assisted review that have a distinctive meaning. Richness, recall, precision, and f-measure are four key terms to know. Richness is the overall percentage of the document population that is responsive to an issue. Recall is the percent of the responsive documents identified by the process and is a measure of completeness. Precision is the percent of documents identified by the process as responsive that are actually responsive and is a measure of quality. F-measure is the weighted average of precision and recall. There are many more to learn, and knowing the vocabulary helps.

4. Apply technology-assisted review before culling with search terms. Using Boolean search terms to cull down the population ahead of assisted review can reduce the overall recall potential. A limitation of Boolean search is that you often find only what you know to look for. Using Boolean search requires the case team to know exactly what language was used in the target documents ahead of time. The strength of technology-assisted review is that it can uncover pockets of relevant content that were not suspected. Most assisted review processes sample across the document universe.

5. Use blind sampling to measure progress and avoid reviewer bias. Many technology-assisted review processes build in sampling to measure progress and provide quality control throughout review. When reviewers are aware that a document is marked responsive or not responsive, it can influence their decision. Representative sampling is a well-established method of determining the characteristics of a large population of documents.

6. Focus on one or two issues at most. Assisted review benefits are highest when ranking documents against the primary issues at the heart of a matter. There are generally dozens of sub-issues. Subdividing responsive documents into the many subcategories is best left to second pass reviewers with explicit review instructions and training.

7. Use second pass linear review. Use a second pass review to identify additional non-responsive documents as well as to identify documents to be withheld for privilege.

Technology-assisted review promises to reduce cost and improve quality compared to other available methods. With the exploding growth of electronically stored information, these new tools are needed. The legal teams that master them first will have a great advantage.

Jon Lavinder

Jon Lavinder, director of technology-assisted review at DTI.

More Posts

Digg This
Reddit This
Stumble Now!
Buzz This
Vote on DZone
Share on Facebook
Bookmark this on Delicious
Kick It on DotNetKicks.com
Shout it
Share on LinkedIn
Bookmark this on Technorati
Post on Twitter
Google Buzz (aka. Google Reader)
PDF24    Send article as PDF   

Filed Under: Technology

About the Author: Jon Lavinder, director of technology-assisted review at DTI.

RSSComments (0)

Trackback URL

Leave a Reply

  • Polls
    Sorry, there are no polls available at the moment.